VYPR
High severityNVD Advisory· Published Feb 9, 2024· Updated Apr 24, 2025

Apache Solr: Backup/Restore APIs allow for deployment of executables in malicious ConfigSets

CVE-2023-50386

Description

Improper Control of Dynamically-Managed Code Resources, Unrestricted Upload of File with Dangerous Type, Inclusion of Functionality from Untrusted Control Sphere vulnerability in Apache Solr.This issue affects Apache Solr: from 6.0.0 through 8.11.2, from 9.0.0 before 9.4.1.

In the affected versions, Solr ConfigSets accepted Java jar and class files to be uploaded through the ConfigSets API. When backing up Solr Collections, these configSet files would be saved to disk when using the LocalFileSystemRepository (the default for backups). If the backup was saved to a directory that Solr uses in its ClassPath/ClassLoaders, then the jar and class files would be available to use with any ConfigSet, trusted or untrusted.

When Solr is run in a secure way (Authorization enabled), as is strongly suggested, this vulnerability is limited to extending the Backup permissions with the ability to add libraries. Users are recommended to upgrade to version 8.11.3 or 9.4.1, which fix the issue. In these versions, the following protections have been added:

  • Users are no longer able to upload files to a configSet that could be executed via a Java ClassLoader.
  • The Backup API restricts saving backups to directories that are used in the ClassLoader.

AI Insight

LLM-synthesized narrative grounded in this CVE's description and references.

CVE-2023-50386 allows uploading malicious Java files via ConfigSets API, which can be executed when backups are saved to a location in Solr's ClassPath.

Vulnerability

Description

CVE-2023-50386 is a critical vulnerability in Apache Solr affecting versions 6.0.0 through 8.11.2 and 9.0.0 before 9.4.1. The flaw stems from improper control of dynamically-managed code resources, unrestricted upload of dangerous file types, and inclusion of functionality from untrusted control spheres. Specifically, the Solr ConfigSets API allowed users to upload Java jar and class files. When backing up Solr Collections using the default LocalFileSystemRepository, these files would be saved to disk. If the backup directory fell within Solr's ClassPath or ClassLoaders, the malicious JAR/class files became available for use with any ConfigSet, including untrusted ones. [1][2]

Attack and

Exploitation

An attacker with access to the ConfigSets API can upload a crafted Java archive or class file. The attack does not require authentication if Solr is not run with Authorization enabled. When a backup operation is performed and saved to a directory that is part of Solr's classpath, the uploaded malicious code is loaded into the JVM. This can be triggered even by trusted ConfigSets that later load the library. In secure deployments with Authorization enabled, the exploitation scope is reduced to users who already have backup permissions, but still allows privilege escalation by adding libraries. [3][4]

Impact

Successful exploitation allows an attacker to execute arbitrary code within the Solr server's Java runtime. This can lead to complete compromise of the Solr instance, including data exfiltration, modification, or denial of service. The vulnerability is particularly severe because it bypasses typical restrictions on code execution in Solr's sandbox.

Mitigation

Apache has released fixes in versions 8.11.3 and 9.4.1. The patches introduce checks that prevent uploading files that could be executed via Java ClassLoader, specifically by validating file content against forbidden MIME types. Additionally, the Backup API now restricts saving backups to directories used in the ClassLoader. Users are strongly advised to upgrade immediately or, if not possible, enable Authorization and limit the exposure of the ConfigSets API. [1][2]

AI Insight generated on May 20, 2026. Synthesized from this CVE's description and the cited reference URLs; citations are validated against the source bundle.

Affected packages

Versions sourced from the GitHub Security Advisory.

PackageAffected versionsPatched versions
org.apache.solr:solr-coreMaven
>= 6.0.0, < 8.11.38.11.3
org.apache.solr:solr-coreMaven
>= 9.0.0, < 9.4.19.4.1

Affected products

7

Patches

4
6c8f24eb9e3f

SOLR-16949: Fix inputstream leaks

https://github.com/apache/lucene-solrJan HøydahlDec 18, 2023via ghsa
2 files changed · +81 49
  • solr/core/src/java/org/apache/solr/util/FileTypeMagicUtil.java+51 21 modified
    @@ -20,7 +20,6 @@
     import com.j256.simplemagic.ContentInfo;
     import com.j256.simplemagic.ContentInfoUtil;
     import com.j256.simplemagic.ContentType;
    -import java.io.ByteArrayInputStream;
     import java.io.IOException;
     import java.io.InputStream;
     import java.nio.file.FileVisitResult;
    @@ -58,42 +57,48 @@ public class FileTypeMagicUtil implements ContentInfoUtil.ErrorCallBack {
       public static void assertConfigSetFolderLegal(Path confPath) throws IOException {
         Files.walkFileTree(confPath, new SimpleFileVisitor<Path>() {
           @Override
    -      public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
    +      public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
             // Read first 100 bytes of the file to determine the mime type
    -        try(InputStream fileStream = Files.newInputStream(file)) {
    -          byte[] bytes = new byte[100];
    -          fileStream.read(bytes);
    -          if (FileTypeMagicUtil.isFileForbiddenInConfigset(bytes)) {
    -            throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
    -                String.format(Locale.ROOT, "Not uploading file %s to configset, as it matched the MAGIC signature of a forbidden mime type %s",
    -                    file, FileTypeMagicUtil.INSTANCE.guessMimeType(bytes)));
    -          }
    -          return FileVisitResult.CONTINUE;
    +        if (FileTypeMagicUtil.isFileForbiddenInConfigset(file)) {
    +          throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
    +              String.format(Locale.ROOT, "Not uploading file %s to configset, as it matched the MAGIC signature of a forbidden mime type %s",
    +                  file, FileTypeMagicUtil.INSTANCE.guessMimeType(file)));
             }
    +        return FileVisitResult.CONTINUE;
           }
     
           @Override
    -      public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
    +      public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) {
             if (SKIP_FOLDERS.contains(dir.getFileName().toString())) return FileVisitResult.SKIP_SUBTREE;
     
             return FileVisitResult.CONTINUE;
           }
         });
       }
     
    +  /**
    +   * Guess the mime type of file based on its magic number.
    +   *
    +   * @param file file to check
    +   * @return string with content-type or "application/octet-stream" if unknown
    +   */
    +  public String guessMimeType(Path file) {
    +    try {
    +      return guessTypeFallbackToOctetStream(util.findMatch(file.toFile()));
    +    } catch (IOException e) {
    +      throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, e);
    +    }
    +  }
    +
       /**
        * Guess the mime type of file based on its magic number.
        *
        * @param stream input stream of the file
        * @return string with content-type or "application/octet-stream" if unknown
        */
    -  public String guessMimeType(InputStream stream) {
    +  String guessMimeType(InputStream stream) {
         try {
    -      ContentInfo info = util.findMatch(stream);
    -      if (info == null) {
    -        return ContentType.OTHER.getMimeType();
    -      }
    -      return info.getContentType().getMimeType();
    +      return guessTypeFallbackToOctetStream(util.findMatch(stream));
         } catch (IOException e) {
           throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, e);
         }
    @@ -106,7 +111,7 @@ public String guessMimeType(InputStream stream) {
        * @return string with content-type or "application/octet-stream" if unknown
        */
       public String guessMimeType(byte[] bytes) {
    -    return guessMimeType(new ByteArrayInputStream(bytes));
    +    return guessTypeFallbackToOctetStream(util.findMatch(bytes));
       }
     
       @Override
    @@ -117,6 +122,24 @@ public void error(String line, String details, Exception e) {
             e);
       }
     
    +  /**
    +   * Determine forbidden file type based on magic bytes matching of the file itself. Forbidden types
    +   * are:
    +   *
    +   * <ul>
    +   *   <li><code>application/x-java-applet</code>: java class file
    +   *   <li><code>application/zip</code>: jar or zip archives
    +   *   <li><code>application/x-tar</code>: tar archives
    +   *   <li><code>text/x-shellscript</code>: shell or bash script
    +   * </ul>
    +   *
    +   * @param file file to check
    +   * @return true if file is among the forbidden mime-types
    +   */
    +  public static boolean isFileForbiddenInConfigset(Path file) {
    +    return forbiddenTypes.contains(FileTypeMagicUtil.INSTANCE.guessMimeType(file));
    +  }
    +
       /**
        * Determine forbidden file type based on magic bytes matching of the file itself. Forbidden types
        * are:
    @@ -131,7 +154,7 @@ public void error(String line, String details, Exception e) {
        * @param fileStream stream from the file content
        * @return true if file is among the forbidden mime-types
        */
    -  public static boolean isFileForbiddenInConfigset(InputStream fileStream) {
    +  static boolean isFileForbiddenInConfigset(InputStream fileStream) {
         return forbiddenTypes.contains(FileTypeMagicUtil.INSTANCE.guessMimeType(fileStream));
       }
     
    @@ -144,7 +167,7 @@ public static boolean isFileForbiddenInConfigset(InputStream fileStream) {
       public static boolean isFileForbiddenInConfigset(byte[] bytes) {
         if (bytes == null || bytes.length == 0)
           return false; // A ZK znode may be a folder with no content
    -    return isFileForbiddenInConfigset(new ByteArrayInputStream(bytes));
    +    return forbiddenTypes.contains(FileTypeMagicUtil.INSTANCE.guessMimeType(bytes));
       }
     
       private static final Set<String> forbiddenTypes =
    @@ -155,4 +178,11 @@ public static boolean isFileForbiddenInConfigset(byte[] bytes) {
                           "application/x-java-applet,application/zip,application/x-tar,text/x-shellscript")
                       .split(",")));
     
    +  private String guessTypeFallbackToOctetStream(ContentInfo contentInfo) {
    +    if (contentInfo == null) {
    +      return ContentType.OTHER.getMimeType();
    +    } else {
    +      return contentInfo.getContentType().getMimeType();
    +    }
    +  }
     }
    
  • solr/core/src/test/org/apache/solr/util/FileTypeMagicUtilTest.java+30 28 modified
    @@ -17,38 +17,40 @@
     
     package org.apache.solr.util;
     
    +import java.io.IOException;
    +import java.io.InputStream;
     import org.apache.solr.SolrTestCaseJ4;
     
     public class FileTypeMagicUtilTest extends SolrTestCaseJ4 {
    -  public void testGuessMimeType() {
    -    assertEquals(
    -        "application/x-java-applet",
    -        FileTypeMagicUtil.INSTANCE.guessMimeType(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")));
    -    assertEquals(
    -        "application/zip",
    -        FileTypeMagicUtil.INSTANCE.guessMimeType(
    -            FileTypeMagicUtil.class.getResourceAsStream(
    -                "/runtimecode/containerplugin.v.1.jar.bin")));
    -    assertEquals(
    -        "application/x-tar",
    -        FileTypeMagicUtil.INSTANCE.guessMimeType(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/hello.tar.bin")));
    -    assertEquals(
    -        "text/x-shellscript",
    -        FileTypeMagicUtil.INSTANCE.guessMimeType(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/shell.sh.txt")));
    +  public void testGuessMimeType() throws IOException {
    +    assertResourceMimeType("application/x-java-applet", "/magic/HelloWorldJavaClass.class.bin");
    +    assertResourceMimeType("application/zip", "/runtimecode/containerplugin.v.1.jar.bin");
    +    assertResourceMimeType("application/x-tar", "/magic/hello.tar.bin");
    +    assertResourceMimeType("text/x-shellscript", "/magic/shell.sh.txt");
       }
     
    -  public void testIsFileForbiddenInConfigset() {
    -    assertTrue(
    -        FileTypeMagicUtil.isFileForbiddenInConfigset(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")));
    -    assertTrue(
    -        FileTypeMagicUtil.isFileForbiddenInConfigset(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/shell.sh.txt")));
    -    assertFalse(
    -        FileTypeMagicUtil.isFileForbiddenInConfigset(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/plain.txt")));
    +  public void testIsFileForbiddenInConfigset() throws IOException {
    +    assertResourceForbiddenInConfigset("/magic/HelloWorldJavaClass.class.bin");
    +    assertResourceForbiddenInConfigset("/magic/shell.sh.txt");
    +    assertResourceAllowedInConfigset("/magic/plain.txt");
    +  }
    +
    +  private void assertResourceMimeType(String mimeType, String resourcePath) throws IOException {
    +    try (InputStream stream =
    +        FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")) {
    +      assertEquals("application/x-java-applet", FileTypeMagicUtil.INSTANCE.guessMimeType(stream));
    +    }
    +  }
    +
    +  private void assertResourceForbiddenInConfigset(String resourcePath) throws IOException {
    +    try (InputStream stream = FileTypeMagicUtil.class.getResourceAsStream(resourcePath)) {
    +      assertTrue(FileTypeMagicUtil.isFileForbiddenInConfigset(stream));
    +    }
    +  }
    +
    +  private void assertResourceAllowedInConfigset(String resourcePath) throws IOException {
    +    try (InputStream stream = FileTypeMagicUtil.class.getResourceAsStream(resourcePath)) {
    +      assertFalse(FileTypeMagicUtil.isFileForbiddenInConfigset(stream));
    +    }
       }
     }
    
c79011e81dad

SOLR-16949: Handle inputStream leaks

https://github.com/apache/solrJan HøydahlDec 17, 2023via ghsa
3 files changed · +96 60
  • solr/core/src/java/org/apache/solr/core/FileSystemConfigSetService.java+2 3 modified
    @@ -214,12 +214,11 @@ public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
                         "Not including uploading file to config, as it is a forbidden type: {}",
                         file.getFileName());
                   } else {
    -                if (!FileTypeMagicUtil.isFileForbiddenInConfigset(Files.newInputStream(file))) {
    +                if (!FileTypeMagicUtil.isFileForbiddenInConfigset(file)) {
                       Files.copy(
                           file, target.resolve(source.relativize(file).toString()), REPLACE_EXISTING);
                     } else {
    -                  String mimeType =
    -                      FileTypeMagicUtil.INSTANCE.guessMimeType(Files.newInputStream(file));
    +                  String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(file);
                       log.warn(
                           "Not copying file {}, as it matched the MAGIC signature of a forbidden mime type {}",
                           file.getFileName(),
    
  • solr/core/src/java/org/apache/solr/util/FileTypeMagicUtil.java+64 29 modified
    @@ -20,7 +20,6 @@
     import com.j256.simplemagic.ContentInfo;
     import com.j256.simplemagic.ContentInfoUtil;
     import com.j256.simplemagic.ContentType;
    -import java.io.ByteArrayInputStream;
     import java.io.IOException;
     import java.io.InputStream;
     import java.nio.file.FileVisitResult;
    @@ -58,30 +57,23 @@ public class FileTypeMagicUtil implements ContentInfoUtil.ErrorCallBack {
       public static void assertConfigSetFolderLegal(Path confPath) throws IOException {
         Files.walkFileTree(
             confPath,
    -        new SimpleFileVisitor<Path>() {
    +        new SimpleFileVisitor<>() {
               @Override
    -          public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
    -              throws IOException {
    -            // Read first 100 bytes of the file to determine the mime type
    -            try (InputStream fileStream = Files.newInputStream(file)) {
    -              byte[] bytes = new byte[100];
    -              fileStream.read(bytes);
    -              if (FileTypeMagicUtil.isFileForbiddenInConfigset(bytes)) {
    -                throw new SolrException(
    -                    SolrException.ErrorCode.BAD_REQUEST,
    -                    String.format(
    -                        Locale.ROOT,
    -                        "Not uploading file %s to configset, as it matched the MAGIC signature of a forbidden mime type %s",
    -                        file,
    -                        FileTypeMagicUtil.INSTANCE.guessMimeType(bytes)));
    -              }
    -              return FileVisitResult.CONTINUE;
    +          public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
    +            if (FileTypeMagicUtil.isFileForbiddenInConfigset(file)) {
    +              throw new SolrException(
    +                  SolrException.ErrorCode.BAD_REQUEST,
    +                  String.format(
    +                      Locale.ROOT,
    +                      "Not uploading file %s to configset, as it matched the MAGIC signature of a forbidden mime type %s",
    +                      file,
    +                      FileTypeMagicUtil.INSTANCE.guessMimeType(file)));
                 }
    +            return FileVisitResult.CONTINUE;
               }
     
               @Override
    -          public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
    -              throws IOException {
    +          public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) {
                 if (SKIP_FOLDERS.contains(dir.getFileName().toString()))
                   return FileVisitResult.SKIP_SUBTREE;
     
    @@ -90,19 +82,29 @@ public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
             });
       }
     
    +  /**
    +   * Guess the mime type of file based on its magic number.
    +   *
    +   * @param file file to check
    +   * @return string with content-type or "application/octet-stream" if unknown
    +   */
    +  public String guessMimeType(Path file) {
    +    try {
    +      return guessTypeFallbackToOctetStream(util.findMatch(file.toFile()));
    +    } catch (IOException e) {
    +      throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, e);
    +    }
    +  }
    +
       /**
        * Guess the mime type of file based on its magic number.
        *
        * @param stream input stream of the file
        * @return string with content-type or "application/octet-stream" if unknown
        */
    -  public String guessMimeType(InputStream stream) {
    +  String guessMimeType(InputStream stream) {
         try {
    -      ContentInfo info = util.findMatch(stream);
    -      if (info == null) {
    -        return ContentType.OTHER.getMimeType();
    -      }
    -      return info.getContentType().getMimeType();
    +      return guessTypeFallbackToOctetStream(util.findMatch(stream));
         } catch (IOException e) {
           throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, e);
         }
    @@ -115,7 +117,7 @@ public String guessMimeType(InputStream stream) {
        * @return string with content-type or "application/octet-stream" if unknown
        */
       public String guessMimeType(byte[] bytes) {
    -    return guessMimeType(new ByteArrayInputStream(bytes));
    +    return guessTypeFallbackToOctetStream(util.findMatch(bytes));
       }
     
       @Override
    @@ -126,6 +128,31 @@ public void error(String line, String details, Exception e) {
             e);
       }
     
    +  /**
    +   * Determine forbidden file type based on magic bytes matching of the file itself. Forbidden types
    +   * are:
    +   *
    +   * <ul>
    +   *   <li><code>application/x-java-applet</code>: java class file
    +   *   <li><code>application/zip</code>: jar or zip archives
    +   *   <li><code>application/x-tar</code>: tar archives
    +   *   <li><code>text/x-shellscript</code>: shell or bash script
    +   * </ul>
    +   *
    +   * @param file file to check
    +   * @return true if file is among the forbidden mime-types
    +   */
    +  public static boolean isFileForbiddenInConfigset(Path file) {
    +    try (InputStream fileStream = Files.newInputStream(file)) {
    +      return isFileForbiddenInConfigset(fileStream);
    +    } catch (IOException e) {
    +      throw new SolrException(
    +          SolrException.ErrorCode.SERVER_ERROR,
    +          String.format(Locale.ROOT, "Error reading file %s", file),
    +          e);
    +    }
    +  }
    +
       /**
        * Determine forbidden file type based on magic bytes matching of the file itself. Forbidden types
        * are:
    @@ -140,7 +167,7 @@ public void error(String line, String details, Exception e) {
        * @param fileStream stream from the file content
        * @return true if file is among the forbidden mime-types
        */
    -  public static boolean isFileForbiddenInConfigset(InputStream fileStream) {
    +  static boolean isFileForbiddenInConfigset(InputStream fileStream) {
         return forbiddenTypes.contains(FileTypeMagicUtil.INSTANCE.guessMimeType(fileStream));
       }
     
    @@ -153,7 +180,7 @@ public static boolean isFileForbiddenInConfigset(InputStream fileStream) {
       public static boolean isFileForbiddenInConfigset(byte[] bytes) {
         if (bytes == null || bytes.length == 0)
           return false; // A ZK znode may be a folder with no content
    -    return isFileForbiddenInConfigset(new ByteArrayInputStream(bytes));
    +    return forbiddenTypes.contains(FileTypeMagicUtil.INSTANCE.guessMimeType(bytes));
       }
     
       private static final Set<String> forbiddenTypes =
    @@ -163,4 +190,12 @@ public static boolean isFileForbiddenInConfigset(byte[] bytes) {
                           "solr.configset.upload.mimetypes.forbidden",
                           "application/x-java-applet,application/zip,application/x-tar,text/x-shellscript")
                       .split(",")));
    +
    +  private String guessTypeFallbackToOctetStream(ContentInfo contentInfo) {
    +    if (contentInfo == null) {
    +      return ContentType.OTHER.getMimeType();
    +    } else {
    +      return contentInfo.getContentType().getMimeType();
    +    }
    +  }
     }
    
  • solr/core/src/test/org/apache/solr/util/FileTypeMagicUtilTest.java+30 28 modified
    @@ -17,38 +17,40 @@
     
     package org.apache.solr.util;
     
    +import java.io.IOException;
    +import java.io.InputStream;
     import org.apache.solr.SolrTestCaseJ4;
     
     public class FileTypeMagicUtilTest extends SolrTestCaseJ4 {
    -  public void testGuessMimeType() {
    -    assertEquals(
    -        "application/x-java-applet",
    -        FileTypeMagicUtil.INSTANCE.guessMimeType(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")));
    -    assertEquals(
    -        "application/zip",
    -        FileTypeMagicUtil.INSTANCE.guessMimeType(
    -            FileTypeMagicUtil.class.getResourceAsStream(
    -                "/runtimecode/containerplugin.v.1.jar.bin")));
    -    assertEquals(
    -        "application/x-tar",
    -        FileTypeMagicUtil.INSTANCE.guessMimeType(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/hello.tar.bin")));
    -    assertEquals(
    -        "text/x-shellscript",
    -        FileTypeMagicUtil.INSTANCE.guessMimeType(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/shell.sh.txt")));
    +  public void testGuessMimeType() throws IOException {
    +    assertResourceMimeType("application/x-java-applet", "/magic/HelloWorldJavaClass.class.bin");
    +    assertResourceMimeType("application/zip", "/runtimecode/containerplugin.v.1.jar.bin");
    +    assertResourceMimeType("application/x-tar", "/magic/hello.tar.bin");
    +    assertResourceMimeType("text/x-shellscript", "/magic/shell.sh.txt");
       }
     
    -  public void testIsFileForbiddenInConfigset() {
    -    assertTrue(
    -        FileTypeMagicUtil.isFileForbiddenInConfigset(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")));
    -    assertTrue(
    -        FileTypeMagicUtil.isFileForbiddenInConfigset(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/shell.sh.txt")));
    -    assertFalse(
    -        FileTypeMagicUtil.isFileForbiddenInConfigset(
    -            FileTypeMagicUtil.class.getResourceAsStream("/magic/plain.txt")));
    +  public void testIsFileForbiddenInConfigset() throws IOException {
    +    assertResourceForbiddenInConfigset("/magic/HelloWorldJavaClass.class.bin");
    +    assertResourceForbiddenInConfigset("/magic/shell.sh.txt");
    +    assertResourceAllowedInConfigset("/magic/plain.txt");
    +  }
    +
    +  private void assertResourceMimeType(String mimeType, String resourcePath) throws IOException {
    +    try (InputStream stream =
    +        FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")) {
    +      assertEquals("application/x-java-applet", FileTypeMagicUtil.INSTANCE.guessMimeType(stream));
    +    }
    +  }
    +
    +  private void assertResourceForbiddenInConfigset(String resourcePath) throws IOException {
    +    try (InputStream stream = FileTypeMagicUtil.class.getResourceAsStream(resourcePath)) {
    +      assertTrue(FileTypeMagicUtil.isFileForbiddenInConfigset(stream));
    +    }
    +  }
    +
    +  private void assertResourceAllowedInConfigset(String resourcePath) throws IOException {
    +    try (InputStream stream = FileTypeMagicUtil.class.getResourceAsStream(resourcePath)) {
    +      assertFalse(FileTypeMagicUtil.isFileForbiddenInConfigset(stream));
    +    }
       }
     }
    
644dd3a6d678

SOLR-16949: Restrict certain file types from being uploaded to or downloaded from Config Sets

https://github.com/apache/solrJan HøydahlDec 13, 2023via ghsa
23 files changed · +522 56
  • solr/CHANGES.txt+2 0 modified
    @@ -104,6 +104,8 @@ Other Changes
     * SOLR-17091: dev tools script cloud.sh became broken after changes in 9.3 added a new -slim.tgz file it was not expecting
       cloud.sh has been updated to ignore the -slim.tgz version of the tarball.
     
    +* SOLR-16949: Restrict certain file types from being uploaded to or downloaded from Config Sets (janhoy, Houston Putman)
    +
     ==================  9.4.0 ==================
     New Features
     ---------------------
    
  • solr/core/build.gradle+2 0 modified
    @@ -161,6 +161,8 @@ dependencies {
     
       compileOnly 'com.github.stephenc.jcip:jcip-annotations'
     
    +  implementation 'com.j256.simplemagic:simplemagic'
    +
       // -- Test Dependencies
     
       testRuntimeOnly 'org.slf4j:jcl-over-slf4j'
    
  • solr/core/src/java/org/apache/solr/cli/ConfigSetUploadTool.java+2 0 modified
    @@ -27,6 +27,7 @@
     import org.apache.solr.common.cloud.SolrZkClient;
     import org.apache.solr.common.cloud.ZkMaintenanceUtils;
     import org.apache.solr.core.ConfigSetService;
    +import org.apache.solr.util.FileTypeMagicUtil;
     import org.slf4j.Logger;
     import org.slf4j.LoggerFactory;
     
    @@ -100,6 +101,7 @@ public void runImpl(CommandLine cli) throws Exception {
                   + cli.getOptionValue("confname")
                   + " to ZooKeeper at "
                   + zkHost);
    +      FileTypeMagicUtil.assertConfigSetFolderLegal(confPath);
           ZkMaintenanceUtils.uploadToZK(
               zkClient,
               confPath,
    
  • solr/core/src/java/org/apache/solr/cloud/ZkConfigSetService.java+20 1 modified
    @@ -22,6 +22,7 @@
     import java.util.ArrayList;
     import java.util.Collections;
     import java.util.List;
    +import java.util.Locale;
     import java.util.Map;
     import java.util.Objects;
     import org.apache.solr.client.solrj.cloud.SolrCloudManager;
    @@ -39,6 +40,7 @@
     import org.apache.solr.core.CoreDescriptor;
     import org.apache.solr.core.SolrConfig;
     import org.apache.solr.core.SolrResourceLoader;
    +import org.apache.solr.util.FileTypeMagicUtil;
     import org.apache.zookeeper.CreateMode;
     import org.apache.zookeeper.KeeperException;
     import org.apache.zookeeper.data.Stat;
    @@ -199,6 +201,15 @@ public void uploadFileToConfig(
         try {
           if (ZkMaintenanceUtils.isFileForbiddenInConfigSets(fileName)) {
             log.warn("Not including uploading file to config, as it is a forbidden type: {}", fileName);
    +      } else if (FileTypeMagicUtil.isFileForbiddenInConfigset(data)) {
    +        String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(data);
    +        throw new SolrException(
    +            SolrException.ErrorCode.BAD_REQUEST,
    +            String.format(
    +                Locale.ROOT,
    +                "Not uploading file %s to config, as it matched the MAGIC signature of a forbidden mime type %s",
    +                fileName,
    +                mimeType));
           } else {
             // if overwriteOnExists is true then zkClient#makePath failOnExists is set to false
             zkClient.makePath(filePath, data, CreateMode.PERSISTENT, null, !overwriteOnExists, true);
    @@ -340,7 +351,15 @@ private void copyData(String fromZkFilePath, String toZkFilePath)
         } else {
           log.debug("Copying zk node {} to {}", fromZkFilePath, toZkFilePath);
           byte[] data = zkClient.getData(fromZkFilePath, null, null, true);
    -      zkClient.makePath(toZkFilePath, data, true);
    +      if (!FileTypeMagicUtil.isFileForbiddenInConfigset(data)) {
    +        zkClient.makePath(toZkFilePath, data, true);
    +      } else {
    +        String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(data);
    +        log.warn(
    +            "Skipping copy of file {} in ZK, as it matched the MAGIC signature of a forbidden mime type {}",
    +            fromZkFilePath,
    +            mimeType);
    +      }
         }
       }
     
    
  • solr/core/src/java/org/apache/solr/core/backup/BackupManager.java+20 3 modified
    @@ -40,6 +40,7 @@
     import org.apache.solr.common.util.Utils;
     import org.apache.solr.core.ConfigSetService;
     import org.apache.solr.core.backup.repository.BackupRepository;
    +import org.apache.solr.util.FileTypeMagicUtil;
     import org.apache.zookeeper.CreateMode;
     import org.apache.zookeeper.KeeperException;
     import org.slf4j.Logger;
    @@ -349,8 +350,16 @@ private void downloadConfigToRepo(ConfigSetService configSetService, String conf
               if (data == null) {
                 data = new byte[0];
               }
    -          try (OutputStream os = repository.createOutput(uri)) {
    -            os.write(data);
    +          if (!FileTypeMagicUtil.isFileForbiddenInConfigset(data)) {
    +            try (OutputStream os = repository.createOutput(uri)) {
    +              os.write(data);
    +            }
    +          } else {
    +            String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(data);
    +            log.warn(
    +                "Not including zookeeper file {} in backup, as it matched the MAGIC signature of a forbidden mime type {}",
    +                filePath,
    +                mimeType);
               }
             }
           } else {
    @@ -379,7 +388,15 @@ private void uploadConfigToSolrCloud(
                     // probably ok since the config file should be small.
                     byte[] arr = new byte[(int) is.length()];
                     is.readBytes(arr, 0, (int) is.length());
    -                configSetService.uploadFileToConfig(configName, filePath, arr, false);
    +                if (!FileTypeMagicUtil.isFileForbiddenInConfigset(arr)) {
    +                  configSetService.uploadFileToConfig(configName, filePath, arr, false);
    +                } else {
    +                  String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(arr);
    +                  log.warn(
    +                      "Not including zookeeper file {} in restore, as it matched the MAGIC signature of a forbidden mime type {}",
    +                      filePath,
    +                      mimeType);
    +                }
                   }
                 }
                 break;
    
  • solr/core/src/java/org/apache/solr/core/FileSystemConfigSetService.java+23 5 modified
    @@ -37,6 +37,7 @@
     import org.apache.solr.common.SolrException;
     import org.apache.solr.common.cloud.ZkMaintenanceUtils;
     import org.apache.solr.common.util.Utils;
    +import org.apache.solr.util.FileTypeMagicUtil;
     import org.slf4j.Logger;
     import org.slf4j.LoggerFactory;
     
    @@ -150,9 +151,17 @@ public void uploadFileToConfig(
         if (ZkMaintenanceUtils.isFileForbiddenInConfigSets(fileName)) {
           log.warn("Not including uploading file to config, as it is a forbidden type: {}", fileName);
         } else {
    -      Path filePath = getConfigDir(configName).resolve(normalizePathToOsSeparator(fileName));
    -      if (!Files.exists(filePath) || overwriteOnExists) {
    -        Files.write(filePath, data);
    +      if (!FileTypeMagicUtil.isFileForbiddenInConfigset(data)) {
    +        Path filePath = getConfigDir(configName).resolve(normalizePathToOsSeparator(fileName));
    +        if (!Files.exists(filePath) || overwriteOnExists) {
    +          Files.write(filePath, data);
    +        }
    +      } else {
    +        String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(data);
    +        log.warn(
    +            "Not including uploading file {}, as it matched the MAGIC signature of a forbidden mime type {}",
    +            fileName,
    +            mimeType);
           }
         }
       }
    @@ -205,8 +214,17 @@ public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
                         "Not including uploading file to config, as it is a forbidden type: {}",
                         file.getFileName());
                   } else {
    -                Files.copy(
    -                    file, target.resolve(source.relativize(file).toString()), REPLACE_EXISTING);
    +                if (!FileTypeMagicUtil.isFileForbiddenInConfigset(Files.newInputStream(file))) {
    +                  Files.copy(
    +                      file, target.resolve(source.relativize(file).toString()), REPLACE_EXISTING);
    +                } else {
    +                  String mimeType =
    +                      FileTypeMagicUtil.INSTANCE.guessMimeType(Files.newInputStream(file));
    +                  log.warn(
    +                      "Not copying file {}, as it matched the MAGIC signature of a forbidden mime type {}",
    +                      file.getFileName(),
    +                      mimeType);
    +                }
                   }
                   return FileVisitResult.CONTINUE;
                 }
    
  • solr/core/src/java/org/apache/solr/handler/configsets/UploadConfigSetFileAPI.java+5 3 modified
    @@ -27,6 +27,7 @@
     import org.apache.solr.core.CoreContainer;
     import org.apache.solr.request.SolrQueryRequest;
     import org.apache.solr.response.SolrQueryResponse;
    +import org.apache.solr.util.FileTypeMagicUtil;
     
     /**
      * V2 API for adding or updating a single file within a configset.
    @@ -67,11 +68,13 @@ public void updateConfigSetFile(SolrQueryRequest req, SolrQueryResponse rsp) thr
         if (fixedSingleFilePath.charAt(0) == '/') {
           fixedSingleFilePath = fixedSingleFilePath.substring(1);
         }
    +    byte[] data = inputStream.readAllBytes();
         if (fixedSingleFilePath.isEmpty()) {
           throw new SolrException(
               SolrException.ErrorCode.BAD_REQUEST,
               "The file path provided for upload, '" + singleFilePath + "', is not valid.");
    -    } else if (ZkMaintenanceUtils.isFileForbiddenInConfigSets(fixedSingleFilePath)) {
    +    } else if (ZkMaintenanceUtils.isFileForbiddenInConfigSets(fixedSingleFilePath)
    +        || FileTypeMagicUtil.isFileForbiddenInConfigset(data)) {
           throw new SolrException(
               SolrException.ErrorCode.BAD_REQUEST,
               "The file type provided for upload, '"
    @@ -87,8 +90,7 @@ public void updateConfigSetFile(SolrQueryRequest req, SolrQueryResponse rsp) thr
           // For creating the baseNode, the cleanup parameter is only allowed to be true when
           // singleFilePath is not passed.
           createBaseNode(configSetService, overwritesExisting, requestIsTrusted, configSetName);
    -      configSetService.uploadFileToConfig(
    -          configSetName, fixedSingleFilePath, inputStream.readAllBytes(), allowOverwrite);
    +      configSetService.uploadFileToConfig(configSetName, fixedSingleFilePath, data, allowOverwrite);
         }
       }
     }
    
  • solr/core/src/java/org/apache/solr/util/FileTypeMagicUtil.java+166 0 added
    @@ -0,0 +1,166 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.solr.util;
    +
    +import com.j256.simplemagic.ContentInfo;
    +import com.j256.simplemagic.ContentInfoUtil;
    +import com.j256.simplemagic.ContentType;
    +import java.io.ByteArrayInputStream;
    +import java.io.IOException;
    +import java.io.InputStream;
    +import java.nio.file.FileVisitResult;
    +import java.nio.file.Files;
    +import java.nio.file.Path;
    +import java.nio.file.SimpleFileVisitor;
    +import java.nio.file.attribute.BasicFileAttributes;
    +import java.util.Arrays;
    +import java.util.HashSet;
    +import java.util.Locale;
    +import java.util.Set;
    +import org.apache.solr.common.SolrException;
    +
    +/** Utility class to guess the mime type of file based on its magic number. */
    +public class FileTypeMagicUtil implements ContentInfoUtil.ErrorCallBack {
    +  private final ContentInfoUtil util;
    +  private static final Set<String> SKIP_FOLDERS = new HashSet<>(Arrays.asList(".", ".."));
    +
    +  public static FileTypeMagicUtil INSTANCE = new FileTypeMagicUtil();
    +
    +  FileTypeMagicUtil() {
    +    try {
    +      util = new ContentInfoUtil("/magic/executables", this);
    +    } catch (IOException e) {
    +      throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error parsing magic file", e);
    +    }
    +  }
    +
    +  /**
    +   * Asserts that an entire configset folder is legal to upload.
    +   *
    +   * @param confPath the path to the folder
    +   * @throws SolrException if an illegal file is found in the folder structure
    +   */
    +  public static void assertConfigSetFolderLegal(Path confPath) throws IOException {
    +    Files.walkFileTree(
    +        confPath,
    +        new SimpleFileVisitor<Path>() {
    +          @Override
    +          public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
    +              throws IOException {
    +            // Read first 100 bytes of the file to determine the mime type
    +            try (InputStream fileStream = Files.newInputStream(file)) {
    +              byte[] bytes = new byte[100];
    +              fileStream.read(bytes);
    +              if (FileTypeMagicUtil.isFileForbiddenInConfigset(bytes)) {
    +                throw new SolrException(
    +                    SolrException.ErrorCode.BAD_REQUEST,
    +                    String.format(
    +                        Locale.ROOT,
    +                        "Not uploading file %s to configset, as it matched the MAGIC signature of a forbidden mime type %s",
    +                        file,
    +                        FileTypeMagicUtil.INSTANCE.guessMimeType(bytes)));
    +              }
    +              return FileVisitResult.CONTINUE;
    +            }
    +          }
    +
    +          @Override
    +          public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
    +              throws IOException {
    +            if (SKIP_FOLDERS.contains(dir.getFileName().toString()))
    +              return FileVisitResult.SKIP_SUBTREE;
    +
    +            return FileVisitResult.CONTINUE;
    +          }
    +        });
    +  }
    +
    +  /**
    +   * Guess the mime type of file based on its magic number.
    +   *
    +   * @param stream input stream of the file
    +   * @return string with content-type or "application/octet-stream" if unknown
    +   */
    +  public String guessMimeType(InputStream stream) {
    +    try {
    +      ContentInfo info = util.findMatch(stream);
    +      if (info == null) {
    +        return ContentType.OTHER.getMimeType();
    +      }
    +      return info.getContentType().getMimeType();
    +    } catch (IOException e) {
    +      throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, e);
    +    }
    +  }
    +
    +  /**
    +   * Guess the mime type of file bytes based on its magic number.
    +   *
    +   * @param bytes the first bytes at start of the file
    +   * @return string with content-type or "application/octet-stream" if unknown
    +   */
    +  public String guessMimeType(byte[] bytes) {
    +    return guessMimeType(new ByteArrayInputStream(bytes));
    +  }
    +
    +  @Override
    +  public void error(String line, String details, Exception e) {
    +    throw new SolrException(
    +        SolrException.ErrorCode.SERVER_ERROR,
    +        String.format(Locale.ROOT, "%s: %s", line, details),
    +        e);
    +  }
    +
    +  /**
    +   * Determine forbidden file type based on magic bytes matching of the file itself. Forbidden types
    +   * are:
    +   *
    +   * <ul>
    +   *   <li><code>application/x-java-applet</code>: java class file
    +   *   <li><code>application/zip</code>: jar or zip archives
    +   *   <li><code>application/x-tar</code>: tar archives
    +   *   <li><code>text/x-shellscript</code>: shell or bash script
    +   * </ul>
    +   *
    +   * @param fileStream stream from the file content
    +   * @return true if file is among the forbidden mime-types
    +   */
    +  public static boolean isFileForbiddenInConfigset(InputStream fileStream) {
    +    return forbiddenTypes.contains(FileTypeMagicUtil.INSTANCE.guessMimeType(fileStream));
    +  }
    +
    +  /**
    +   * Determine forbidden file type based on magic bytes matching of the first bytes of the file.
    +   *
    +   * @param bytes byte array of the file content
    +   * @return true if file is among the forbidden mime-types
    +   */
    +  public static boolean isFileForbiddenInConfigset(byte[] bytes) {
    +    if (bytes == null || bytes.length == 0)
    +      return false; // A ZK znode may be a folder with no content
    +    return isFileForbiddenInConfigset(new ByteArrayInputStream(bytes));
    +  }
    +
    +  private static final Set<String> forbiddenTypes =
    +      new HashSet<>(
    +          Arrays.asList(
    +              System.getProperty(
    +                      "solr.configset.upload.mimetypes.forbidden",
    +                      "application/x-java-applet,application/zip,application/x-tar,text/x-shellscript")
    +                  .split(",")));
    +}
    
  • solr/core/src/resources/magic/executables+74 0 added
    @@ -0,0 +1,74 @@
    +#  Licensed to the Apache Software Foundation (ASF) under one or more
    +#  contributor license agreements.  See the NOTICE file distributed with
    +#  this work for additional information regarding copyright ownership.
    +#  The ASF licenses this file to You under the Apache License, Version 2.0
    +#  (the "License"); you may not use this file except in compliance with
    +#  the License.  You may obtain a copy of the License at
    +#
    +#       http://www.apache.org/licenses/LICENSE-2.0
    +#
    +#  Unless required by applicable law or agreed to in writing, software
    +#  distributed under the License is distributed on an "AS IS" BASIS,
    +#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    +#  See the License for the specific language governing permissions and
    +#  limitations under the License.
    +
    +# POSIX tar archives
    +# URL: https://en.wikipedia.org/wiki/Tar_(computing)
    +# Reference: https://www.freebsd.org/cgi/man.cgi?query=tar&sektion=5&manpath=FreeBSD+8-current
    +# header mainly padded with nul bytes
    +500	quad		0
    +!:strength /2
    +# filename or extended attribute printable strings in range space null til umlaut ue
    +>0	ubeshort	>0x1F00
    +>>0	ubeshort	<0xFCFD
    +# last 4 header bytes often null but tar\0 in gtarfail2.tar gtarfail.tar-bad
    +# at https://sourceforge.net/projects/s-tar/files/testscripts/
    +>>>508	ubelong&0x8B9E8DFF	0
    +# nul, space or ascii digit 0-7 at start of mode
    +>>>>100	ubyte&0xC8	=0
    +>>>>>101 ubyte&0xC8	=0
    +# nul, space at end of check sum
    +>>>>>>155 ubyte&0xDF	=0
    +# space or ascii digit 0 at start of check sum
    +>>>>>>>148	ubyte&0xEF	=0x20
    +# check for specific 1st member name that indicates other mime type and file name suffix
    +>>>>>>>>0	string		TpmEmuTpms/permall
    +!:mime	application/x-tar
    +!:ext	tar
    +# other stuff in padding
    +# some implementations add new fields to the blank area at the end of the header record
    +# created for example by DOS TAR 3.20g 1994 Tim V.Shapore with -j option
    +>>257	ulong		!0		tar archive (old)
    +!:mime	application/x-tar
    +!:ext	tar
    +# magic in newer, GNU, posix variants
    +>257	string		=ustar
    +# 2 last char of magic and UStar version because string expression does not work
    +# 2 space characters followed by a null for GNU variant
    +>>261	ubelong		=0x72202000	POSIX tar archive (GNU)
    +!:mime	application/x-gtar
    +!:ext	tar/gtar
    +
    +
    +# Zip archives (Greg Roelofs, c/o zip-bugs@wkuvx1.wku.edu)
    +0	string		PK\005\006	Zip archive data (empty)
    +0	string		PK\003\004  Zip archive data
    +!:strength +1
    +!:mime application/zip
    +!:ext zip/cbz
    +
    +
    +# JAVA
    +0	belong		0xcafebabe
    +>4	ubelong		>30		compiled Java class data,
    +!:mime	application/x-java-applet
    +#!:mime	application/java-byte-code
    +!:ext	class
    +
    +
    +# SHELL scripts
    +#0	string/w	:			shell archive or script for antique kernel text
    +0	regex	\^#!\\s?(/bin/|/usr/)		POSIX shell script text executable
    +!:mime	text/x-shellscript
    +!:ext	sh/bash
    \ No newline at end of file
    
  • solr/core/src/test-files/magic/hello.tar.bin+0 0 added
  • solr/core/src/test-files/magic/HelloWorldJavaClass.class.bin+0 0 added
  • solr/core/src/test-files/magic/HelloWorld.java.txt+5 0 added
    @@ -0,0 +1,5 @@
    +class HelloWorld {
    +  public static void main(String[] args) {
    +    System.out.println("Hellow world");
    +  }
    +}
    \ No newline at end of file
    
  • solr/core/src/test-files/magic/plain.txt+1 0 added
    @@ -0,0 +1 @@
    +Hello world
    \ No newline at end of file
    
  • solr/core/src/test-files/magic/README.md+29 0 added
    @@ -0,0 +1,29 @@
    +<!--
    +    Licensed to the Apache Software Foundation (ASF) under one or more
    +    contributor license agreements.  See the NOTICE file distributed with
    +    this work for additional information regarding copyright ownership.
    +    The ASF licenses this file to You under the Apache License, Version 2.0
    +    the "License"); you may not use this file except in compliance with
    +    the License.  You may obtain a copy of the License at
    +
    +        http://www.apache.org/licenses/LICENSE-2.0
    +
    +    Unless required by applicable law or agreed to in writing, software
    +    distributed under the License is distributed on an "AS IS" BASIS,
    +    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    +    See the License for the specific language governing permissions and
    +    limitations under the License.
    + -->
    +
    +The two binary files were created by the following commands:
    +
    +```bash
    +echo "Hello" > hello.txt && \
    +  tar -cvf hello.tar.bin hello.txt && \
    +  rm hello.txt
    +
    +cp HelloWorld.java.txt HelloWorld.java && \
    +  javac HelloWorld.java && \
    +  mv HelloWorld.class HelloWorldJavaClass.class.bin && \
    +  rm HelloWorld.java
    +```
    \ No newline at end of file
    
  • solr/core/src/test-files/magic/shell.sh.txt+2 0 added
    @@ -0,0 +1,2 @@
    +#! /usr/bin/env bash
    +echo Hello
    \ No newline at end of file
    
  • solr/core/src/test/org/apache/solr/cloud/TestConfigSetsAPI.java+97 44 modified
    @@ -45,6 +45,7 @@
     import java.util.List;
     import java.util.Locale;
     import java.util.Map;
    +import java.util.Objects;
     import java.util.Properties;
     import java.util.Set;
     import java.util.concurrent.TimeUnit;
    @@ -592,14 +593,14 @@ public void testOverwrite(boolean v2) throws Exception {
           assertEquals(
               "Can't overwrite an existing configset unless the overwrite parameter is set",
               400,
    -          uploadConfigSet(configsetName, configsetSuffix, null, false, false, v2, false));
    +          uploadConfigSet(configsetName, configsetSuffix, null, false, false, v2, false, false));
           unIgnoreException("The configuration regulartestOverwrite-1 already exists in zookeeper");
           assertEquals(
               "Expecting version to remain equal",
               solrconfigZkVersion,
               getConfigZNodeVersion(zkClient, configsetName, configsetSuffix, "solrconfig.xml"));
           assertEquals(
    -          0, uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, false));
    +          0, uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, false, false));
           assertTrue(
               "Expecting version bump",
               solrconfigZkVersion
    @@ -638,13 +639,14 @@ public void testOverwriteWithCleanup(boolean v2) throws Exception {
             zkClient.makePath(f, true);
           }
           assertEquals(
    -          0, uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, false));
    +          0, uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, false, false));
           for (String f : extraFiles) {
             assertTrue(
                 "Expecting file " + f + " to exist in ConfigSet but it's gone",
                 zkClient.exists(f, true));
           }
    -      assertEquals(0, uploadConfigSet(configsetName, configsetSuffix, null, true, true, v2, false));
    +      assertEquals(
    +          0, uploadConfigSet(configsetName, configsetSuffix, null, true, true, v2, false, false));
           for (String f : extraFiles) {
             assertFalse(
                 "Expecting file " + f + " to be deleted from ConfigSet but it wasn't",
    @@ -675,7 +677,8 @@ public void testOverwriteWithForbiddenFiles(boolean v2) throws Exception {
                 .withConnTimeOut(45000, TimeUnit.MILLISECONDS)
                 .build()) {
           String configPath = "/configs/" + configsetName + configsetSuffix;
    -      assertEquals(0, uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, true));
    +      assertEquals(
    +          0, uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, true, false));
           for (String fileEnding : ZkMaintenanceUtils.DEFAULT_FORBIDDEN_FILE_TYPES) {
             String f = configPath + "/test." + fileEnding;
             assertFalse(
    @@ -710,7 +713,7 @@ public void testOverwriteWithTrust(boolean v2) throws Exception {
               getConfigZNodeVersion(zkClient, configsetName, configsetSuffix, "solrconfig.xml");
           // Was untrusted, overwrite with untrusted
           assertEquals(
    -          0, uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, false));
    +          0, uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, false, false));
           assertTrue(
               "Expecting version bump",
               solrconfigZkVersion
    @@ -721,7 +724,8 @@ public void testOverwriteWithTrust(boolean v2) throws Exception {
     
           // Was untrusted, overwrite with trusted but no cleanup
           assertEquals(
    -          0, uploadConfigSet(configsetName, configsetSuffix, "solr", true, false, v2, false));
    +          0,
    +          uploadConfigSet(configsetName, configsetSuffix, "solr", true, false, v2, false, false));
           assertTrue(
               "Expecting version bump",
               solrconfigZkVersion
    @@ -747,7 +751,7 @@ public void testOverwriteWithTrust(boolean v2) throws Exception {
     
           // Was untrusted, overwrite with trusted with cleanup
           assertEquals(
    -          0, uploadConfigSet(configsetName, configsetSuffix, "solr", true, true, v2, false));
    +          0, uploadConfigSet(configsetName, configsetSuffix, "solr", true, true, v2, false, false));
           assertTrue(
               "Expecting version bump",
               solrconfigZkVersion
    @@ -761,7 +765,7 @@ public void testOverwriteWithTrust(boolean v2) throws Exception {
           assertEquals(
               "Can't upload a trusted configset with an untrusted request",
               400,
    -          uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, false));
    +          uploadConfigSet(configsetName, configsetSuffix, null, true, false, v2, false, false));
           assertEquals(
               "Expecting version to remain equal",
               solrconfigZkVersion,
    @@ -773,7 +777,7 @@ public void testOverwriteWithTrust(boolean v2) throws Exception {
           assertEquals(
               "Can't upload a trusted configset with an untrusted request",
               400,
    -          uploadConfigSet(configsetName, configsetSuffix, null, true, true, v2, false));
    +          uploadConfigSet(configsetName, configsetSuffix, null, true, true, v2, false, false));
           assertEquals(
               "Expecting version to remain equal",
               solrconfigZkVersion,
    @@ -783,7 +787,8 @@ public void testOverwriteWithTrust(boolean v2) throws Exception {
     
           // Was trusted, overwrite with trusted no cleanup
           assertEquals(
    -          0, uploadConfigSet(configsetName, configsetSuffix, "solr", true, false, v2, false));
    +          0,
    +          uploadConfigSet(configsetName, configsetSuffix, "solr", true, false, v2, false, false));
           assertTrue(
               "Expecting version bump",
               solrconfigZkVersion
    @@ -794,7 +799,7 @@ public void testOverwriteWithTrust(boolean v2) throws Exception {
     
           // Was trusted, overwrite with trusted with cleanup
           assertEquals(
    -          0, uploadConfigSet(configsetName, configsetSuffix, "solr", true, true, v2, false));
    +          0, uploadConfigSet(configsetName, configsetSuffix, "solr", true, true, v2, false, false));
           assertTrue(
               "Expecting version bump",
               solrconfigZkVersion
    @@ -1457,6 +1462,13 @@ public void testUploadWithLibDirective() throws Exception {
                 .get("id"));
       }
     
    +  @Test
    +  public void testUploadWithForbiddenContent() throws Exception {
    +    // Uploads a config set containing a script, a class file and jar file, will return 400 error
    +    long res = uploadConfigSet("forbidden", "suffix", "foo", true, false, true, false, true);
    +    assertEquals(400, res);
    +  }
    +
       private static String getSecurityJson() {
         return "{\n"
             + "  'authentication':{\n"
    @@ -1511,7 +1523,7 @@ private long uploadConfigSet(
           String configSetName, String suffix, String username, SolrZkClient zkClient, boolean v2)
           throws IOException {
         assertFalse(getConfigSetService().checkConfigExists(configSetName + suffix));
    -    return uploadConfigSet(configSetName, suffix, username, false, false, v2, false);
    +    return uploadConfigSet(configSetName, suffix, username, false, false, v2, false, false);
       }
     
       private long uploadConfigSet(
    @@ -1521,21 +1533,25 @@ private long uploadConfigSet(
           boolean overwrite,
           boolean cleanup,
           boolean v2,
    -      boolean forbiddenTypes)
    +      boolean forbiddenTypes,
    +      boolean forbiddenContent)
           throws IOException {
     
    +    File zipFile;
    +    if (forbiddenTypes) {
    +      log.info("Uploading configset with forbidden file endings");
    +      zipFile =
    +          createTempZipFileWithForbiddenTypes(
    +              "solr/configsets/upload/" + configSetName + "/solrconfig.xml");
    +    } else if (forbiddenContent) {
    +      log.info("Uploading configset with forbidden file content");
    +      zipFile = createTempZipFileWithForbiddenContent("magic");
    +    } else {
    +      zipFile = createTempZipFile("solr/configsets/upload/" + configSetName);
    +    }
    +
         // Read zipped sample config
    -    return uploadGivenConfigSet(
    -        forbiddenTypes
    -            ? createTempZipFileWithForbiddenTypes(
    -                "solr/configsets/upload/" + configSetName + "/solrconfig.xml")
    -            : createTempZipFile("solr/configsets/upload/" + configSetName),
    -        configSetName,
    -        suffix,
    -        username,
    -        overwrite,
    -        cleanup,
    -        v2);
    +    return uploadGivenConfigSet(zipFile, configSetName, suffix, username, overwrite, cleanup, v2);
       }
     
       private long uploadBadConfigSet(String configSetName, String suffix, String username, boolean v2)
    @@ -1702,31 +1718,68 @@ private File createTempZipFileWithForbiddenTypes(String file) {
         }
       }
     
    -  private static void zipWithForbiddenEndings(File file, File zipfile) throws IOException {
    -    OutputStream out = new FileOutputStream(zipfile);
    -    ZipOutputStream zout = new ZipOutputStream(out);
    +  /** Create a zip file (in the temp directory) containing files with forbidden content */
    +  private File createTempZipFileWithForbiddenContent(String resourcePath) {
         try {
    -      for (String fileType : ZkMaintenanceUtils.DEFAULT_FORBIDDEN_FILE_TYPES) {
    -        zout.putNextEntry(new ZipEntry("test." + fileType));
    +      final File zipFile = createTempFile("configset", "zip").toFile();
    +      final File directory = SolrTestCaseJ4.getFile(resourcePath);
    +      if (log.isInfoEnabled()) {
    +        log.info("Directory: {}", directory.getAbsolutePath());
    +      }
    +      zipWithForbiddenContent(directory, zipFile);
    +      if (log.isInfoEnabled()) {
    +        log.info("Zipfile: {}", zipFile.getAbsolutePath());
    +      }
    +      return zipFile;
    +    } catch (IOException e) {
    +      throw new RuntimeException(e);
    +    }
    +  }
     
    -        InputStream in = new FileInputStream(file);
    -        try {
    -          byte[] buffer = new byte[1024];
    -          while (true) {
    -            int readCount = in.read(buffer);
    -            if (readCount < 0) {
    -              break;
    +  private static void zipWithForbiddenContent(File directory, File zipfile) throws IOException {
    +    OutputStream out = Files.newOutputStream(zipfile.toPath());
    +    assertTrue(directory.isDirectory());
    +    try (ZipOutputStream zout = new ZipOutputStream(out)) {
    +      // Copy in all files from the directory
    +      for (File file : Objects.requireNonNull(directory.listFiles())) {
    +        zout.putNextEntry(new ZipEntry(file.getName()));
    +        zout.write(Files.readAllBytes(file.toPath()));
    +        zout.closeEntry();
    +      }
    +    }
    +  }
    +
    +  private static void zipWithForbiddenEndings(File fileOrDirectory, File zipfile)
    +      throws IOException {
    +    OutputStream out = new FileOutputStream(zipfile);
    +    try (ZipOutputStream zout = new ZipOutputStream(out)) {
    +      if (fileOrDirectory.isFile()) {
    +        // Create entries with given file, one for each forbidden endding
    +        for (String fileType : ZkMaintenanceUtils.DEFAULT_FORBIDDEN_FILE_TYPES) {
    +          zout.putNextEntry(new ZipEntry("test." + fileType));
    +
    +          try (InputStream in = new FileInputStream(fileOrDirectory)) {
    +            byte[] buffer = new byte[1024];
    +            while (true) {
    +              int readCount = in.read(buffer);
    +              if (readCount < 0) {
    +                break;
    +              }
    +              zout.write(buffer, 0, readCount);
                 }
    -            zout.write(buffer, 0, readCount);
               }
    -        } finally {
    -          in.close();
    -        }
     
    -        zout.closeEntry();
    +          zout.closeEntry();
    +        }
    +      }
    +      if (fileOrDirectory.isDirectory()) {
    +        // Copy in all files from the directory
    +        for (File file : Objects.requireNonNull(fileOrDirectory.listFiles())) {
    +          zout.putNextEntry(new ZipEntry(file.getName()));
    +          zout.write(Files.readAllBytes(file.toPath()));
    +          zout.closeEntry();
    +        }
           }
    -    } finally {
    -      zout.close();
         }
       }
     
    
  • solr/core/src/test/org/apache/solr/util/FileTypeMagicUtilTest.java+54 0 added
    @@ -0,0 +1,54 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.solr.util;
    +
    +import org.apache.solr.SolrTestCaseJ4;
    +
    +public class FileTypeMagicUtilTest extends SolrTestCaseJ4 {
    +  public void testGuessMimeType() {
    +    assertEquals(
    +        "application/x-java-applet",
    +        FileTypeMagicUtil.INSTANCE.guessMimeType(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")));
    +    assertEquals(
    +        "application/zip",
    +        FileTypeMagicUtil.INSTANCE.guessMimeType(
    +            FileTypeMagicUtil.class.getResourceAsStream(
    +                "/runtimecode/containerplugin.v.1.jar.bin")));
    +    assertEquals(
    +        "application/x-tar",
    +        FileTypeMagicUtil.INSTANCE.guessMimeType(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/hello.tar.bin")));
    +    assertEquals(
    +        "text/x-shellscript",
    +        FileTypeMagicUtil.INSTANCE.guessMimeType(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/shell.sh.txt")));
    +  }
    +
    +  public void testIsFileForbiddenInConfigset() {
    +    assertTrue(
    +        FileTypeMagicUtil.isFileForbiddenInConfigset(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")));
    +    assertTrue(
    +        FileTypeMagicUtil.isFileForbiddenInConfigset(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/shell.sh.txt")));
    +    assertFalse(
    +        FileTypeMagicUtil.isFileForbiddenInConfigset(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/plain.txt")));
    +  }
    +}
    
  • solr/licenses/simplemagic-1.17.jar.sha1+1 0 added
    @@ -0,0 +1 @@
    +b6e2d1e47d7172e57fa858a2e3940c09a590e61e
    
  • solr/licenses/simplemagic-LICENSE-BSD_LIKE.txt+15 0 added
    @@ -0,0 +1,15 @@
    +ISC License (https://opensource.org/licenses/ISC)
    +
    +Copyright 2021, Gray Watson
    +
    +Permission to use, copy, modify, and/or distribute this software for any
    +purpose with or without fee is hereby granted, provided that the above
    +copyright notice and this permission notice appear in all copies.
    +
    +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
    +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
    +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
    +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
    +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
    +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
    +OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
    \ No newline at end of file
    
  • solr/licenses/simplemagic-NOTICE.txt+0 0 added
  • solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkMaintenanceUtils.java+2 0 modified
    @@ -348,6 +348,7 @@ public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
                       USE_FORBIDDEN_FILE_TYPES);
                   return FileVisitResult.CONTINUE;
                 }
    +            // TODO: Cannot check MAGIC header for file since FileTypeGuesser is in core
                 String zkNode = createZkNodeName(zkPath, rootPath, file);
                 try {
                   // if the path exists (and presumably we're uploading data to it) just set its data
    @@ -437,6 +438,7 @@ public static void downloadFromZK(SolrZkClient zkClient, String zkPath, Path fil
             if (isFileForbiddenInConfigSets(zkPath)) {
               log.warn("Skipping download of file from ZK, as it is a forbidden type: {}", zkPath);
             } else {
    +          // TODO: Cannot check MAGIC header for file since FileTypeGuesser is in core
               if (copyDataDown(zkClient, zkPath, file) == 0) {
                 Files.createFile(file);
               }
    
  • versions.lock+1 0 modified
    @@ -62,6 +62,7 @@ com.googlecode.plist:dd-plist:1.24 (1 constraints: 300c84f5)
     com.healthmarketscience.jackcess:jackcess:4.0.2 (1 constraints: 5d0cf201)
     com.healthmarketscience.jackcess:jackcess-encrypt:4.0.1 (1 constraints: 5c0cf101)
     com.ibm.icu:icu4j:70.1 (1 constraints: a90f1784)
    +com.j256.simplemagic:simplemagic:1.17 (1 constraints: dd04f830)
     com.jayway.jsonpath:json-path:2.8.0 (2 constraints: 6c12952c)
     com.lmax:disruptor:3.4.4 (1 constraints: 0d050a36)
     com.mchange:c3p0:0.9.5.5 (1 constraints: c80c571b)
    
  • versions.props+1 0 modified
    @@ -13,6 +13,7 @@ com.google.cloud:google-cloud-bom=0.204.0
     com.google.errorprone:*=2.23.0
     com.google.guava:guava=32.1.3-jre
     com.google.re2j:re2j=1.7
    +com.j256.simplemagic:simplemagic=1.17
     com.jayway.jsonpath:json-path=2.8.0
     com.lmax:disruptor=3.4.4
     com.tdunning:t-digest=3.1
    
7e9a2e67f812

SOLR-16949: Restrict certain file types from being uploaded to or downloaded from Config Sets

https://github.com/apache/lucene-solrJan HøydahlDec 13, 2023via ghsa
20 files changed · +410 11
  • lucene/ivy-versions.properties+1 0 modified
    @@ -71,6 +71,7 @@ com.fasterxml.jackson.core.version = 2.15.2
     /com.healthmarketscience.jackcess/jackcess = 3.0.1
     /com.healthmarketscience.jackcess/jackcess-encrypt = 3.0.0
     /com.ibm.icu/icu4j = 62.1
    +/com.j256.simplemagic/simplemagic = 1.17
     /com.jayway.jsonpath/json-path = 2.7.0
     /com.lmax/disruptor = 3.4.2
     /com.pff/java-libpst = 0.9.3
    
  • solr/CHANGES.txt+2 0 modified
    @@ -64,6 +64,8 @@ Other Changes
     * SOLR-14853: Security: Converted enableRemoteStreaming and enableStreamBody solrconfig options into system properties and env vars.
       Attempts to set them the old way are no-op and log a warning. (David Smiley, janhoy, Ishan Chattopadhyaya)
     
    +* SOLR-16949: Restrict certain file types from being uploaded to or downloaded from Config Sets (janhoy, Houston Putman)
    +
     ==================  8.11.2 ==================
     
     Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
    
  • solr/core/ivy.xml+1 0 modified
    @@ -45,6 +45,7 @@
         <dependency org="com.google.guava" name="failureaccess" rev="${/com.google.guava/failureaccess}" conf="compile"/>
         <dependency org="com.google.guava" name="listenablefuture" rev="${/com.google.guava/listenablefuture}" conf="compile"/>
         <dependency org="com.google.j2objc" name="j2objc-annotations" rev="${/com.google.j2objc/j2objc-annotations}" conf="compile"/>
    +    <dependency org="com.j256.simplemagic" name="simplemagic" rev="${/com.j256.simplemagic/simplemagic}" conf="compile"/>
         <dependency org="org.locationtech.spatial4j" name="spatial4j" rev="${/org.locationtech.spatial4j/spatial4j}" conf="compile"/>
         <dependency org="org.antlr" name="antlr4-runtime" rev="${/org.antlr/antlr4-runtime}"/>
         <dependency org="org.apache.commons" name="commons-math3" rev="${/org.apache.commons/commons-math3}" conf="compile"/>
    
  • solr/core/src/java/org/apache/solr/core/backup/BackupManager.java+20 3 modified
    @@ -27,6 +27,7 @@
     import java.util.List;
     import java.util.Objects;
     import java.util.Optional;
    +import org.apache.solr.util.FileTypeMagicUtil;
     
     import com.google.common.base.Preconditions;
     import org.apache.lucene.store.IOContext;
    @@ -302,8 +303,16 @@ private void downloadFromZK(SolrZkClient zkClient, String zkPath, URI dir) throw
             if (children.size() == 0) {
               log.debug("Writing file {}", file);
               byte[] data = zkClient.getData(zkPath + "/" + file, null, null, true);
    -          try (OutputStream os = repository.createOutput(repository.resolve(dir, file))) {
    -            os.write(data);
    +          if (!FileTypeMagicUtil.isFileForbiddenInConfigset(data)) {
    +            try (OutputStream os = repository.createOutput(repository.resolve(dir, file))) {
    +              os.write(data);
    +            }
    +          } else {
    +            String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(data);
    +            log.warn(
    +                "Not including zookeeper file {} in backup, as it matched the MAGIC signature of a forbidden mime type {}",
    +                file,
    +                mimeType);
               }
             } else {
               URI uri = repository.resolve(dir, file);
    @@ -329,7 +338,15 @@ private void uploadToZk(SolrZkClient zkClient, URI sourceDir, String destZkPath)
               try (IndexInput is = repository.openInput(sourceDir, file, IOContext.DEFAULT)) {
                 byte[] arr = new byte[(int) is.length()]; // probably ok since the config file should be small.
                 is.readBytes(arr, 0, (int) is.length());
    -            zkClient.makePath(zkNodePath, arr, true);
    +            if (!FileTypeMagicUtil.isFileForbiddenInConfigset(arr)) {
    +              zkClient.makePath(zkNodePath, arr, true);
    +            } else {
    +              String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(arr);
    +              log.warn(
    +                  "Not restoring configset file {} to zookeeper, as it matched the MAGIC signature of a forbidden mime type {}",
    +                  file,
    +                  mimeType);
    +            }
               } catch (KeeperException | InterruptedException e) {
                 throw new IOException(SolrZkClient.checkInterrupted(e));
               }
    
  • solr/core/src/java/org/apache/solr/handler/admin/ConfigSetsHandler.java+22 5 modified
    @@ -24,6 +24,7 @@
     import java.util.HashSet;
     import java.util.Iterator;
     import java.util.List;
    +import java.util.Locale;
     import java.util.Map;
     import java.util.Set;
     import java.util.concurrent.TimeUnit;
    @@ -56,6 +57,7 @@
     import org.apache.solr.security.AuthenticationPlugin;
     import org.apache.solr.security.AuthorizationContext;
     import org.apache.solr.security.PermissionNameProvider;
    +import org.apache.solr.util.FileTypeMagicUtil;
     import org.apache.zookeeper.CreateMode;
     import org.apache.zookeeper.KeeperException;
     import org.slf4j.Logger;
    @@ -203,9 +205,17 @@ private void handleConfigUploadRequest(SolrQueryRequest req, SolrQueryResponse r
             try {
               // Create a node for the configuration in zookeeper
               // For creating the baseZnode, the cleanup parameter is only allowed to be true when singleFilePath is not passed.
    -          createBaseZnode(zkClient, overwritesExisting, requestIsTrusted, configPathInZk);
    -          String filePathInZk = configPathInZk + "/" + fixedSingleFilePath;
    -          zkClient.makePath(filePathInZk, IOUtils.toByteArray(inputStream), CreateMode.PERSISTENT, null, !allowOverwrite, true);
    +          byte[] bytes = IOUtils.toByteArray(inputStream);
    +          if (!FileTypeMagicUtil.isFileForbiddenInConfigset(bytes)) {
    +            createBaseZnode(zkClient, overwritesExisting, requestIsTrusted, configPathInZk);
    +            String filePathInZk = configPathInZk + "/" + fixedSingleFilePath;
    +            zkClient.makePath(filePathInZk, bytes, CreateMode.PERSISTENT, null, !allowOverwrite, true);
    +          } else {
    +            String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(bytes);
    +            throw new SolrException(ErrorCode.BAD_REQUEST,
    +                String.format(Locale.ROOT, "Not uploading file %s to configset, as it matched the MAGIC signature of a forbidden mime type %s",
    +                    fixedSingleFilePath, mimeType));
    +          }
             } catch(KeeperException.NodeExistsException nodeExistsException) {
               throw new SolrException(ErrorCode.BAD_REQUEST,
                       "The path " + singleFilePath + " for configSet " + configSetName + " already exists. In order to overwrite, provide overwrite=true or use an HTTP PUT with the V2 API.");
    @@ -244,8 +254,15 @@ private void handleConfigUploadRequest(SolrQueryRequest req, SolrQueryResponse r
           if (zipEntry.isDirectory()) {
             zkClient.makePath(filePathInZk, false,  true);
           } else {
    -        createZkNodeIfNotExistsAndSetData(zkClient, filePathInZk,
    -            IOUtils.toByteArray(zis));
    +        byte[] bytes = IOUtils.toByteArray(zis);
    +        if (!FileTypeMagicUtil.isFileForbiddenInConfigset(bytes)) {
    +          createZkNodeIfNotExistsAndSetData(zkClient, filePathInZk, bytes);
    +        } else {
    +          String mimeType = FileTypeMagicUtil.INSTANCE.guessMimeType(bytes);
    +          throw new SolrException(ErrorCode.BAD_REQUEST,
    +              String.format(Locale.ROOT, "Not uploading file %s to configset, as it matched the MAGIC signature of a forbidden mime type %s",
    +                  zipEntry.getName(), mimeType));
    +        }
           }
         }
         zis.close();
    
  • solr/core/src/java/org/apache/solr/util/FileTypeMagicUtil.java+156 0 added
    @@ -0,0 +1,156 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.solr.util;
    +
    +import com.j256.simplemagic.ContentInfo;
    +import com.j256.simplemagic.ContentInfoUtil;
    +import com.j256.simplemagic.ContentType;
    +import java.io.ByteArrayInputStream;
    +import java.io.IOException;
    +import java.io.InputStream;
    +import java.nio.file.FileVisitResult;
    +import java.nio.file.Files;
    +import java.nio.file.Path;
    +import java.nio.file.SimpleFileVisitor;
    +import java.nio.file.attribute.BasicFileAttributes;
    +import java.util.Arrays;
    +import java.util.HashSet;
    +import java.util.Locale;
    +import java.util.Set;
    +import org.apache.solr.common.SolrException;
    +
    +/** Utility class to guess the mime type of file based on its magic number. */
    +public class FileTypeMagicUtil implements ContentInfoUtil.ErrorCallBack {
    +  private final ContentInfoUtil util;
    +  private static final Set<String> SKIP_FOLDERS = new HashSet<>(Arrays.asList(".", ".."));
    +
    +  public static FileTypeMagicUtil INSTANCE = new FileTypeMagicUtil();
    +
    +  FileTypeMagicUtil() {
    +    try {
    +      util = new ContentInfoUtil("/magic/executables", this);
    +    } catch (IOException e) {
    +      throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error parsing magic file", e);
    +    }
    +  }
    +
    +  /**
    +   * Asserts that an entire configset folder is legal to upload.
    +   *
    +   * @param confPath the path to the folder
    +   * @throws SolrException if an illegal file is found in the folder structure
    +   */
    +  public static void assertConfigSetFolderLegal(Path confPath) throws IOException {
    +    Files.walkFileTree(confPath, new SimpleFileVisitor<Path>() {
    +      @Override
    +      public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
    +        // Read first 100 bytes of the file to determine the mime type
    +        try(InputStream fileStream = Files.newInputStream(file)) {
    +          byte[] bytes = new byte[100];
    +          fileStream.read(bytes);
    +          if (FileTypeMagicUtil.isFileForbiddenInConfigset(bytes)) {
    +            throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
    +                String.format(Locale.ROOT, "Not uploading file %s to configset, as it matched the MAGIC signature of a forbidden mime type %s",
    +                    file, FileTypeMagicUtil.INSTANCE.guessMimeType(bytes)));
    +          }
    +          return FileVisitResult.CONTINUE;
    +        }
    +      }
    +
    +      @Override
    +      public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
    +        if (SKIP_FOLDERS.contains(dir.getFileName().toString())) return FileVisitResult.SKIP_SUBTREE;
    +
    +        return FileVisitResult.CONTINUE;
    +      }
    +    });
    +  }
    +
    +  /**
    +   * Guess the mime type of file based on its magic number.
    +   *
    +   * @param stream input stream of the file
    +   * @return string with content-type or "application/octet-stream" if unknown
    +   */
    +  public String guessMimeType(InputStream stream) {
    +    try {
    +      ContentInfo info = util.findMatch(stream);
    +      if (info == null) {
    +        return ContentType.OTHER.getMimeType();
    +      }
    +      return info.getContentType().getMimeType();
    +    } catch (IOException e) {
    +      throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, e);
    +    }
    +  }
    +
    +  /**
    +   * Guess the mime type of file bytes based on its magic number.
    +   *
    +   * @param bytes the first bytes at start of the file
    +   * @return string with content-type or "application/octet-stream" if unknown
    +   */
    +  public String guessMimeType(byte[] bytes) {
    +    return guessMimeType(new ByteArrayInputStream(bytes));
    +  }
    +
    +  @Override
    +  public void error(String line, String details, Exception e) {
    +    throw new SolrException(
    +        SolrException.ErrorCode.SERVER_ERROR,
    +        String.format(Locale.ROOT, "%s: %s", line, details),
    +        e);
    +  }
    +
    +  /**
    +   * Determine forbidden file type based on magic bytes matching of the file itself. Forbidden types
    +   * are:
    +   *
    +   * <ul>
    +   *   <li><code>application/x-java-applet</code>: java class file
    +   *   <li><code>application/zip</code>: jar or zip archives
    +   *   <li><code>application/x-tar</code>: tar archives
    +   *   <li><code>text/x-shellscript</code>: shell or bash script
    +   * </ul>
    +   *
    +   * @param fileStream stream from the file content
    +   * @return true if file is among the forbidden mime-types
    +   */
    +  public static boolean isFileForbiddenInConfigset(InputStream fileStream) {
    +    return forbiddenTypes.contains(FileTypeMagicUtil.INSTANCE.guessMimeType(fileStream));
    +  }
    +
    +  /**
    +   * Determine forbidden file type based on magic bytes matching of the first bytes of the file.
    +   *
    +   * @param bytes byte array of the file content
    +   * @return true if file is among the forbidden mime-types
    +   */
    +  public static boolean isFileForbiddenInConfigset(byte[] bytes) {
    +    return isFileForbiddenInConfigset(new ByteArrayInputStream(bytes));
    +  }
    +
    +  private static final Set<String> forbiddenTypes =
    +      new HashSet<>(
    +          Arrays.asList(
    +              System.getProperty(
    +                      "solr.configset.upload.mimetypes.forbidden",
    +                      "application/x-java-applet,application/zip,application/x-tar,text/x-shellscript")
    +                  .split(",")));
    +
    +}
    
  • solr/core/src/java/org/apache/solr/util/SolrCLI.java+2 0 modified
    @@ -1971,6 +1971,7 @@ protected void runCloudTool(CloudSolrClient cloudSolrClient, CommandLine cli) th
     
             echoIfVerbose("Uploading " + confPath.toAbsolutePath().toString() +
                 " for config " + confname + " to ZooKeeper at " + cloudSolrClient.getZkHost(), cli);
    +        FileTypeMagicUtil.assertConfigSetFolderLegal(confPath);
             ((ZkClientClusterStateProvider) cloudSolrClient.getClusterStateProvider()).uploadConfig(confPath, confname);
           }
     
    @@ -2265,6 +2266,7 @@ protected void runImpl(CommandLine cli) throws Exception {
             echo("Uploading " + confPath.toAbsolutePath().toString() +
                 " for config " + cli.getOptionValue("confname") + " to ZooKeeper at " + zkHost);
     
    +        FileTypeMagicUtil.assertConfigSetFolderLegal(confPath);
             zkClient.upConfig(confPath, confName);
           } catch (Exception e) {
             log.error("Could not complete upconfig operation for reason: ", e);
    
  • solr/core/src/resources/magic/executables+59 0 added
    @@ -0,0 +1,59 @@
    +# POSIX tar archives
    +# URL: https://en.wikipedia.org/wiki/Tar_(computing)
    +# Reference: https://www.freebsd.org/cgi/man.cgi?query=tar&sektion=5&manpath=FreeBSD+8-current
    +# header mainly padded with nul bytes
    +500	quad		0
    +!:strength /2
    +# filename or extended attribute printable strings in range space null til umlaut ue
    +>0	ubeshort	>0x1F00
    +>>0	ubeshort	<0xFCFD
    +# last 4 header bytes often null but tar\0 in gtarfail2.tar gtarfail.tar-bad
    +# at https://sourceforge.net/projects/s-tar/files/testscripts/
    +>>>508	ubelong&0x8B9E8DFF	0
    +# nul, space or ascii digit 0-7 at start of mode
    +>>>>100	ubyte&0xC8	=0
    +>>>>>101 ubyte&0xC8	=0
    +# nul, space at end of check sum
    +>>>>>>155 ubyte&0xDF	=0
    +# space or ascii digit 0 at start of check sum
    +>>>>>>>148	ubyte&0xEF	=0x20
    +# check for specific 1st member name that indicates other mime type and file name suffix
    +>>>>>>>>0	string		TpmEmuTpms/permall
    +!:mime	application/x-tar
    +!:ext	tar
    +# other stuff in padding
    +# some implementations add new fields to the blank area at the end of the header record
    +# created for example by DOS TAR 3.20g 1994 Tim V.Shapore with -j option
    +>>257	ulong		!0		tar archive (old)
    +!:mime	application/x-tar
    +!:ext	tar
    +# magic in newer, GNU, posix variants
    +>257	string		=ustar
    +# 2 last char of magic and UStar version because string expression does not work
    +# 2 space characters followed by a null for GNU variant
    +>>261	ubelong		=0x72202000	POSIX tar archive (GNU)
    +!:mime	application/x-gtar
    +!:ext	tar/gtar
    +
    +
    +# Zip archives (Greg Roelofs, c/o zip-bugs@wkuvx1.wku.edu)
    +0	string		PK\005\006	Zip archive data (empty)
    +0	string		PK\003\004  Zip archive data
    +!:strength +1
    +!:mime application/zip
    +!:ext zip/cbz
    +
    +
    +# JAVA
    +0	belong		0xcafebabe
    +>4	ubelong		>30		compiled Java class data,
    +!:mime	application/x-java-applet
    +#!:mime	application/java-byte-code
    +!:ext	class
    +
    +
    +# SHELL scripts
    +#0	string/w	:			shell archive or script for antique kernel text
    +0	regex	\^#!\\s?(/bin/|/usr/)		POSIX shell script text executable
    +!:mime	text/x-shellscript
    +!:ext	sh/bash
    \ No newline at end of file
    
  • solr/core/src/test-files/magic/hello.tar.bin+0 0 added
  • solr/core/src/test-files/magic/HelloWorldJavaClass.class.bin+0 0 added
  • solr/core/src/test-files/magic/HelloWorld.java.txt+5 0 added
    @@ -0,0 +1,5 @@
    +class HelloWorld {
    +  public static void main(String[] args) {
    +    System.out.println("Hellow world");
    +  }
    +}
    \ No newline at end of file
    
  • solr/core/src/test-files/magic/plain.txt+1 0 added
    @@ -0,0 +1 @@
    +Hello world
    \ No newline at end of file
    
  • solr/core/src/test-files/magic/README.md+12 0 added
    @@ -0,0 +1,12 @@
    +The two binary files were created by the following commands:
    +
    +```bash
    +echo "Hello" > hello.txt && \
    +  tar -cvf hello.tar.bin hello.txt && \
    +  rm hello.txt
    +
    +cp HelloWorld.java.txt HelloWorld.java && \
    +  javac HelloWorld.java && \
    +  mv HelloWorld.class HelloWorldJavaClass.class.bin && \
    +  rm HelloWorld.java
    +```
    \ No newline at end of file
    
  • solr/core/src/test-files/magic/shell.sh.txt+2 0 added
    @@ -0,0 +1,2 @@
    +#! /usr/bin/env bash
    +echo Hello
    \ No newline at end of file
    
  • solr/core/src/test/org/apache/solr/cloud/TestConfigSetsAPI.java+55 3 modified
    @@ -22,7 +22,6 @@
     import javax.servlet.ServletResponse;
     import javax.servlet.http.HttpServletRequest;
     import javax.servlet.http.HttpServletRequestWrapper;
    -
     import java.io.ByteArrayInputStream;
     import java.io.File;
     import java.io.FileInputStream;
    @@ -34,6 +33,7 @@
     import java.lang.invoke.MethodHandles;
     import java.net.URI;
     import java.nio.ByteBuffer;
    +import java.nio.file.Files;
     import java.security.Principal;
     import java.util.Arrays;
     import java.util.Collection;
    @@ -46,6 +46,7 @@
     import java.util.List;
     import java.util.Locale;
     import java.util.Map;
    +import java.util.Objects;
     import java.util.Properties;
     import java.util.Set;
     import java.util.zip.ZipEntry;
    @@ -90,17 +91,17 @@
     import org.apache.solr.common.util.ValidatingJsonMap;
     import org.apache.solr.core.ConfigSetProperties;
     import org.apache.solr.core.TestDynamicLoading;
    -import org.apache.solr.security.BasicAuthPlugin;
     import org.apache.solr.security.AuthorizationContext;
     import org.apache.solr.security.AuthorizationPlugin;
     import org.apache.solr.security.AuthorizationResponse;
    +import org.apache.solr.security.BasicAuthPlugin;
     import org.apache.solr.servlet.SolrDispatchFilter;
     import org.apache.solr.util.ExternalPaths;
     import org.apache.zookeeper.CreateMode;
     import org.apache.zookeeper.KeeperException;
    +import org.apache.zookeeper.data.Stat;
     import org.junit.After;
     import org.junit.AfterClass;
    -import org.apache.zookeeper.data.Stat;
     import org.junit.Assume;
     import org.junit.BeforeClass;
     import org.junit.Test;
    @@ -903,6 +904,19 @@ public void testUploadWithLibDirective() throws Exception {
             params("q", "*:*")).getResults().get(0).get("id"));
       }
     
    +  @Test
    +  public void testUploadWithForbiddenContent() throws Exception {
    +    // Uploads a config set containing a script, a class file and jar file, will return 400 error
    +    long res = uploadForbiddenConfigSet(
    +        "forbidden",
    +        "suffix",
    +        "foo",
    +        true,
    +        false,
    +        true);
    +    assertEquals(400, res);
    +  }
    +
       private static String getSecurityJson() throws KeeperException, InterruptedException {
         String securityJson = "{\n" +
                 "  'authentication':{\n" +
    @@ -955,6 +969,12 @@ private long uploadConfigSet(String configSetName, String suffix, String usernam
                                     configSetName, suffix, username, overwrite, cleanup, v2);
       }
     
    +  private long uploadForbiddenConfigSet(String configSetName, String suffix, String username,
    +                                        boolean overwrite, boolean cleanup, boolean v2) throws IOException {
    +    return uploadGivenConfigSet(createTempZipFileWithForbiddenContent("magic"),
    +        configSetName, suffix, username, overwrite, cleanup, v2);
    +  }
    +
       private long uploadBadConfigSet(String configSetName, String suffix, String username,
                                       boolean overwrite, boolean cleanup, boolean v2) throws IOException {
         
    @@ -1319,6 +1339,38 @@ private StringBuilder getConfigSetProps(Map<String, String> map) {
         return new StringBuilder(new String(Utils.toJSON(map), UTF_8));
       }
     
    +  /** Create a zip file (in the temp directory) containing files with forbidden content */
    +  private File createTempZipFileWithForbiddenContent(String resourcePath) {
    +    try {
    +      final File zipFile = createTempFile("configset", "zip").toFile();
    +      final File directory = SolrTestCaseJ4.getFile(resourcePath);
    +      if (log.isInfoEnabled()) {
    +        log.info("Directory: {}", directory.getAbsolutePath());
    +      }
    +      zipWithForbiddenContent(directory, zipFile);
    +      if (log.isInfoEnabled()) {
    +        log.info("Zipfile: {}", zipFile.getAbsolutePath());
    +      }
    +      return zipFile;
    +    } catch (IOException e) {
    +      throw new RuntimeException(e);
    +    }
    +  }
    +
    +  private static void zipWithForbiddenContent(File directory, File zipfile)
    +      throws IOException {
    +    OutputStream out = Files.newOutputStream(zipfile.toPath());
    +    assertTrue(directory.isDirectory());
    +    try (ZipOutputStream zout = new ZipOutputStream(out)) {
    +      // Copy in all files from the directory
    +      for (File file : Objects.requireNonNull(directory.listFiles())) {
    +        zout.putNextEntry(new ZipEntry(file.getName()));
    +        zout.write(Files.readAllBytes(file.toPath()));
    +        zout.closeEntry();
    +      }
    +    }
    +  }
    +
       public static class CreateNoErrorChecking extends ConfigSetAdminRequest.Create {
         @SuppressWarnings({"rawtypes"})
         public ConfigSetAdminRequest setAction(ConfigSetAction action) {
    
  • solr/core/src/test/org/apache/solr/util/FileTypeMagicUtilTest.java+54 0 added
    @@ -0,0 +1,54 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.solr.util;
    +
    +import org.apache.solr.SolrTestCaseJ4;
    +
    +public class FileTypeMagicUtilTest extends SolrTestCaseJ4 {
    +  public void testGuessMimeType() {
    +    assertEquals(
    +        "application/x-java-applet",
    +        FileTypeMagicUtil.INSTANCE.guessMimeType(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")));
    +    assertEquals(
    +        "application/zip",
    +        FileTypeMagicUtil.INSTANCE.guessMimeType(
    +            FileTypeMagicUtil.class.getResourceAsStream(
    +                "/runtimecode/containerplugin.v.1.jar.bin")));
    +    assertEquals(
    +        "application/x-tar",
    +        FileTypeMagicUtil.INSTANCE.guessMimeType(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/hello.tar.bin")));
    +    assertEquals(
    +        "text/x-shellscript",
    +        FileTypeMagicUtil.INSTANCE.guessMimeType(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/shell.sh.txt")));
    +  }
    +
    +  public void testIsFileForbiddenInConfigset() {
    +    assertTrue(
    +        FileTypeMagicUtil.isFileForbiddenInConfigset(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/HelloWorldJavaClass.class.bin")));
    +    assertTrue(
    +        FileTypeMagicUtil.isFileForbiddenInConfigset(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/shell.sh.txt")));
    +    assertFalse(
    +        FileTypeMagicUtil.isFileForbiddenInConfigset(
    +            FileTypeMagicUtil.class.getResourceAsStream("/magic/plain.txt")));
    +  }
    +}
    
  • solr/licenses/simplemagic-1.17.jar.sha1+1 0 added
    @@ -0,0 +1 @@
    +b6e2d1e47d7172e57fa858a2e3940c09a590e61e
    
  • solr/licenses/simplemagic-LICENSE-BSD_LIKE.txt+15 0 added
    @@ -0,0 +1,15 @@
    +ISC License (https://opensource.org/licenses/ISC)
    +
    +Copyright 2021, Gray Watson
    +
    +Permission to use, copy, modify, and/or distribute this software for any
    +purpose with or without fee is hereby granted, provided that the above
    +copyright notice and this permission notice appear in all copies.
    +
    +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
    +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
    +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
    +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
    +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
    +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
    +OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
    \ No newline at end of file
    
  • solr/licenses/simplemagic-NOTICE.txt+0 0 added
  • solr/solrj/src/java/org/apache/solr/common/cloud/ZkMaintenanceUtils.java+2 0 modified
    @@ -301,6 +301,7 @@ public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IO
               log.info("uploadToZK skipping '{}' due to filenameExclusions '{}'", filename, filenameExclusions);
               return FileVisitResult.CONTINUE;
             }
    +        // TODO: Cannot check MAGIC header for file since FileTypeMagicUtil is in core
             String zkNode = createZkNodeName(zkPath, rootPath, file);
             try {
               // if the path exists (and presumably we're uploading data to it) just set its data
    @@ -348,6 +349,7 @@ public static void downloadFromZK(SolrZkClient zkClient, String zkPath, Path fil
           if (children.size() == 0) {
             // If we didn't copy data down, then we also didn't create the file. But we still need a marker on the local
             // disk so create an empty file.
    +        // TODO: Cannot check MAGIC header for file since FileTypeGuesser is in core
             if (copyDataDown(zkClient, zkPath, file.toFile()) == 0) {
               Files.createFile(file);
             }
    

Vulnerability mechanics

Generated on May 9, 2026. Inputs: CWE entries + fix-commit diffs from this CVE's patches. Citations validated against bundle.

References

9

News mentions

0

No linked articles in our index yet.