CrateDB database has an arbitrary file read vulnerability
Description
CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of data in real-time. There is a COPY FROM function in the CrateDB database that is used to import file data into database tables. This function has a flaw, and authenticated attackers can use the COPY FROM function to import arbitrary file content into database tables, resulting in information leakage. This vulnerability is patched in 5.3.9, 5.4.8, 5.5.4, and 5.6.1.
AI Insight
LLM-synthesized narrative grounded in this CVE's description and references.
Authenticated CrateDB users could use COPY FROM to read arbitrary files on the host, leading to information disclosure; fixed in versions 5.3.9, 5.4.8, 5.5.4, and 5.6.1.
Vulnerability
Overview
The COPY FROM command in CrateDB, a distributed SQL database, contains a security flaw that allows authenticated users to import arbitrary file content into database tables. The root cause is that the command did not properly restrict access to local file URIs for non-superuser accounts, enabling any user with credentials for the cluster to read files from the host filesystem where the CrateDB process runs.[1][2]
Exploitation
Conditions
An attacker only needs valid authentication credentials for the CrateDB database — no special privileges are required. Using the COPY FROM statement with a file:// URI (e.g., COPY table FROM '/etc/passwd'), an attacker can read any file that the operating system user running the CrateDB process has read access to. The attack is straightforward and requires no complex chain of actions.[1][2]
Impact
Successful exploitation leads to local arbitrary file disclosure, potentially exposing sensitive data such as configuration files, credentials, cryptographic keys, or other secrets stored on the server. This vulnerability is categorized as an information leakage issue, with a CVSS severity that reflects the high potential for data exposure.[3]
Mitigation
Status
The vulnerability has been patched in CrateDB versions 5.3.9, 5.4.8, 5.5.4, and 5.6.1. The fix restricts use of the file:// scheme in COPY FROM to the crate superuser only, preventing normal users from reading local files. Users should upgrade to one of the patched versions immediately.[1][2][3]
AI Insight generated on May 20, 2026. Synthesized from this CVE's description and the cited reference URLs; citations are validated against the source bundle.
Affected packages
Versions sourced from the GitHub Security Advisory.
| Package | Affected versions | Patched versions |
|---|---|---|
io.crate:crateMaven | < 5.3.9 | 5.3.9 |
io.crate:crateMaven | >= 5.4.0, < 5.4.8 | 5.4.8 |
io.crate:crateMaven | >= 5.5.0, < 5.5.4 | 5.5.4 |
io.crate:crateMaven | >= 5.6.0, < 5.6.1 | 5.6.1 |
Affected products
2- crate/cratev5Range: < 5.3.9
Patches
54e857d675683Restrict `COPY FROM` using local files to superuser
13 files changed · +94 −14
docs/appendices/release-notes/5.5.4.rst+8 −0 modified@@ -44,6 +44,14 @@ Version 5.5.4 - Unreleased See the :ref:`version_5.5.0` release notes for a full list of changes in the 5.5 series. +Security Fixes +============== + +- Fixed a security issue where any CrateDB user could read/import the content of + any file on the host system, the CrateDB process user has read access to, by + using the ``COPY FROM`` command with a file URI. This access is now restricted + to the ``crate`` superuser only. + Fixes =====
docs/appendices/release-notes/5.6.1.rst+7 −0 modified@@ -43,6 +43,13 @@ Version 5.6.1 - Unreleased See the :ref:`version_5.6.0` release notes for a full list of changes in the 5.6 series. +Security Fixes +============== + +- Fixed a security issue where any CrateDB user could read/import the content of + any file on the host system, the CrateDB process user has read access to, by + using the ``COPY FROM`` command with a file URI. This access is now restricted + to the ``crate`` superuser only. Fixes =====
docs/sql/statements/copy-from.rst+2 −0 modified@@ -211,6 +211,8 @@ For example: The files must be accessible on at least one node and the system user running the ``crate`` process must have read access to every file specified. +Additionally, only the ``crate`` superuser is allowed to use the ``file://`` +scheme. By default, every node will attempt to import every file. If the file is accessible on multiple nodes, you can set the `shared`_ option to true in order
plugins/cr8-copy-s3/src/test/java/io/crate/copy/s3/S3FileReadingCollectorTest.java+1 −1 modified@@ -106,7 +106,7 @@ public void testCollectWithOneSocketTimeout() throws Throwable { private FileReadingIterator createBatchIterator(S3ObjectInputStream inputStream, String ... fileUris) { String compression = null; return new FileReadingIterator( - Arrays.asList(fileUris), + Arrays.stream(fileUris).map(FileReadingIterator::toURI).toList(), compression, Map.of( S3FileInputFactory.NAME,
server/src/main/java/io/crate/exceptions/UnauthorizedException.java+9 −1 modified@@ -21,10 +21,18 @@ package io.crate.exceptions; -public class UnauthorizedException extends RuntimeException implements UnscopedException { +import java.io.IOException; + +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.common.io.stream.StreamInput; + +public class UnauthorizedException extends ElasticsearchException implements UnscopedException { public UnauthorizedException(String message) { super(message); } + public UnauthorizedException(StreamInput in) throws IOException { + super(in); + } }
server/src/main/java/io/crate/execution/engine/collect/files/FileReadingIterator.java+2 −3 modified@@ -175,7 +175,7 @@ public boolean equals(Object obj) { } } - public FileReadingIterator(Collection<String> fileUris, + public FileReadingIterator(Collection<URI> fileUris, String compression, Map<String, FileInputFactory> fileInputFactories, Boolean shared, @@ -398,8 +398,7 @@ public static URI toURI(String fileUri) { } @Nullable - private FileInput toFileInput(String fileUri, Settings withClauseOptions) { - URI uri = toURI(fileUri); + private FileInput toFileInput(URI uri, Settings withClauseOptions) { FileInputFactory fileInputFactory = fileInputFactories.get(uri.getScheme()); if (fileInputFactory != null) { try {
server/src/main/java/io/crate/execution/engine/collect/sources/FileCollectSource.java+20 −2 modified@@ -21,6 +21,9 @@ package io.crate.execution.engine.collect.sources; +import static java.util.Objects.requireNonNull; + +import java.net.URI; import java.util.Arrays; import java.util.Collection; import java.util.Collections; @@ -40,6 +43,7 @@ import io.crate.data.BatchIterator; import io.crate.data.Row; import io.crate.data.SkippingBatchIterator; +import io.crate.exceptions.UnauthorizedException; import io.crate.execution.dsl.phases.CollectPhase; import io.crate.execution.dsl.phases.FileUriCollectPhase; import io.crate.execution.engine.collect.CollectTask; @@ -53,6 +57,8 @@ import io.crate.metadata.NodeContext; import io.crate.metadata.TransactionContext; import io.crate.planner.operators.SubQueryResults; +import io.crate.role.Role; +import io.crate.role.Roles; import io.crate.types.DataTypes; @Singleton @@ -63,17 +69,20 @@ public class FileCollectSource implements CollectSource { private final InputFactory inputFactory; private final NodeContext nodeCtx; private final ThreadPool threadPool; + private final Roles roles; @Inject public FileCollectSource(NodeContext nodeCtx, ClusterService clusterService, Map<String, FileInputFactory> fileInputFactoryMap, - ThreadPool threadPool) { + ThreadPool threadPool, + Roles roles) { this.fileInputFactoryMap = fileInputFactoryMap; this.nodeCtx = nodeCtx; this.inputFactory = new InputFactory(nodeCtx); this.clusterService = clusterService; this.threadPool = threadPool; + this.roles = roles; } @Override @@ -86,7 +95,16 @@ public CompletableFuture<BatchIterator<Row>> getIterator(TransactionContext txnC inputFactory.ctxForRefs(txnCtx, FileLineReferenceResolver::getImplementation); ctx.add(collectPhase.toCollect()); - List<String> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()); + Role user = requireNonNull(roles.findUser(txnCtx.sessionSettings().userName()), "User who invoked a statement must exist"); + List<URI> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()).stream() + .map(s -> { + var uri = FileReadingIterator.toURI(s); + if (uri.getScheme().equals("file") && user.isSuperUser() == false) { + throw new UnauthorizedException("Only a superuser can read from the local file system"); + } + return uri; + }) + .toList(); FileReadingIterator fileReadingIterator = new FileReadingIterator( fileUris, fileUriCollectPhase.compression(),
server/src/main/java/org/elasticsearch/ElasticsearchException.java+6 −1 modified@@ -973,7 +973,12 @@ private enum ElasticsearchExceptionHandle { io.crate.exceptions.OperationOnInaccessibleRelationException.class, io.crate.exceptions.OperationOnInaccessibleRelationException::new, 176, - Version.V_5_6_0); + Version.V_5_6_0), + UNAUTHORIZED_EXCEPTION( + io.crate.exceptions.UnauthorizedException.class, + io.crate.exceptions.UnauthorizedException::new, + 177, + Version.V_5_7_0); final Class<? extends ElasticsearchException> exceptionClass; final CheckedFunction<StreamInput, ? extends ElasticsearchException, IOException> constructor;
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingCollectorTest.java+1 −1 modified@@ -200,7 +200,7 @@ private static FileReadingIterator it(String ... fileUris) { private static FileReadingIterator it(Collection<String> fileUris, String compression) { return new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), compression, Map.of(LocalFsFileInputFactory.NAME, new LocalFsFileInputFactory()), false,
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingIteratorTest.java+9 −3 modified@@ -34,6 +34,7 @@ import java.io.InputStream; import java.io.InputStreamReader; import java.net.SocketTimeoutException; +import java.net.URI; import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; @@ -43,6 +44,7 @@ import java.util.concurrent.ScheduledExecutorService; import java.util.concurrent.TimeUnit; import java.util.function.Supplier; +import java.util.stream.Stream; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.test.ESTestCase; @@ -87,7 +89,9 @@ public void test_iterator_closes_current_reader_on_io_error() throws Exception { Path tempFile2 = createTempFile("tempfile2", ".csv"); List<String> lines2 = List.of("name,id,age", "Trillian,5,33"); Files.write(tempFile2, lines2); - List<String> fileUris = List.of(tempFile1.toUri().toString(), tempFile2.toUri().toString()); + List<URI> fileUris = Stream.of(tempFile1.toUri().toString(), tempFile2.toUri().toString()) + .map(FileReadingIterator::toURI) + .toList(); Supplier<BatchIterator<LineCursor>> batchIteratorSupplier = () -> new FileReadingIterator( @@ -139,7 +143,8 @@ public void test_consecutive_retries_will_not_result_in_duplicate_reads() throws Path tempFile = createTempFile("tempfile1", ".csv"); List<String> lines = List.of("id", "1", "2", "3", "4", "5"); Files.write(tempFile, lines); - List<String> fileUris = List.of(tempFile.toUri().toString()); + List<URI> fileUris = Stream.of(tempFile.toUri().toString()) + .map(FileReadingIterator::toURI).toList(); Supplier<BatchIterator<LineCursor>> batchIteratorSupplier = () -> new FileReadingIterator( @@ -213,7 +218,8 @@ public void test_retry_from_one_uri_does_not_affect_reading_next_uri() throws Ex Files.write(tempFile, List.of("1", "2", "3")); Path tempFile2 = createTempFile("tempfile2", ".csv"); Files.write(tempFile2, List.of("4", "5", "6")); - List<String> fileUris = List.of(tempFile.toUri().toString(), tempFile2.toUri().toString()); + List<URI> fileUris = Stream.of(tempFile.toUri().toString(), tempFile2.toUri().toString()) + .map(FileReadingIterator::toURI).toList(); var fi = new FileReadingIterator( fileUris,
server/src/test/java/io/crate/execution/engine/collect/MapSideDataCollectOperationTest.java+3 −1 modified@@ -54,6 +54,7 @@ import io.crate.expression.symbol.Literal; import io.crate.metadata.ColumnIdent; import io.crate.metadata.CoordinatorTxnCtx; +import io.crate.role.Role; import io.crate.test.integration.CrateDummyClusterServiceUnitTest; import io.crate.types.DataTypes; @@ -69,7 +70,8 @@ public void testFileUriCollect() throws Exception { createNodeContext(), clusterService, Collections.emptyMap(), - THREAD_POOL + THREAD_POOL, + () -> List.of(Role.CRATE_USER) ); File tmpFile = temporaryFolder.newFile("fileUriCollectOperation.json");
server/src/test/java/io/crate/execution/engine/collect/sources/FileCollectSourceTest.java+2 −1 modified@@ -92,7 +92,8 @@ public void test_file_collect_source_returns_iterator_that_can_skip_lines() thro new NodeContext(new Functions(Map.of()), roles), clusterService, Map.of(), - THREAD_POOL + THREAD_POOL, + () -> List.of(Role.CRATE_USER) ); CompletableFuture<BatchIterator<Row>> iterator = fileCollectSource.getIterator(
server/src/test/java/io/crate/integrationtests/CopyIntegrationTest.java+24 −0 modified@@ -26,6 +26,7 @@ import static io.crate.testing.Asserts.assertThat; import static io.crate.testing.TestingHelpers.printedTable; import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST; +import static org.assertj.core.api.Assertions.assertThatThrownBy; import java.io.File; import java.io.FileOutputStream; @@ -53,10 +54,15 @@ import com.carrotsearch.randomizedtesting.LifecycleScope; +import io.crate.action.sql.Sessions; +import io.crate.exceptions.UnauthorizedException; +import io.crate.role.Role; +import io.crate.role.Roles; import io.crate.testing.Asserts; import io.crate.testing.SQLResponse; import io.crate.testing.UseJdbc; import io.crate.testing.UseNewCluster; +import io.crate.testing.UseRandomizedSchema; @IntegTestCase.ClusterScope(numDataNodes = 2) public class CopyIntegrationTest extends SQLHttpIntegrationTest { @@ -1196,4 +1202,22 @@ primary key (id) "2| 31123| apple safari| 23073" ); } + + @UseRandomizedSchema(random = false) + @Test + public void test_copy_from_local_file_is_only_allowed_for_superusers() { + execute("CREATE TABLE quotes (id INT PRIMARY KEY, " + + "quote STRING INDEX USING FULLTEXT) WITH (number_of_replicas = 0)"); + execute("CREATE USER test_user"); + execute("GRANT ALL TO test_user"); + + var roles = cluster().getInstance(Roles.class); + Role user = roles.findUser("test_user"); + Sessions sqlOperations = cluster().getInstance(Sessions.class); + try (var session = sqlOperations.newSession(null, user)) { + assertThatThrownBy(() -> execute("COPY quotes FROM ?", new Object[]{copyFilePath + "test_copy_from.json"}, session)) + .isExactlyInstanceOf(UnauthorizedException.class) + .hasMessage("Only a superuser can read from the local file system"); + } + } }
c4c97d5a1c52Restrict `COPY FROM` using local files to superuser
13 files changed · +98 −28
benchmarks/src/main/java/io/crate/execution/engine/reader/CsvReaderBenchmark.java+1 −1 modified@@ -141,7 +141,7 @@ public void measureFileReadingIteratorForCSV(Blackhole blackhole) { List<Input<?>> inputs = Collections.singletonList(ctx.add(raw)); BatchIterator<Row> batchIterator = FileReadingIterator.newInstance( - Collections.singletonList(fileUri), + List.of(FileReadingIterator.toURI(fileUri)), inputs, ctx.expressions(), null,
benchmarks/src/main/java/io/crate/execution/engine/reader/JsonReaderBenchmark.java+1 −1 modified@@ -141,7 +141,7 @@ public void measureFileReadingIteratorForJson(Blackhole blackhole) { List<Input<?>> inputs = Collections.singletonList(ctx.add(raw)); BatchIterator<Row> batchIterator = FileReadingIterator.newInstance( - Collections.singletonList(fileUri), + List.of(FileReadingIterator.toURI(fileUri)), inputs, ctx.expressions(), null,
docs/appendices/release-notes/5.3.9.rst+8 −0 modified@@ -44,6 +44,14 @@ Version 5.3.9 - Unreleased See the :ref:`version_5.3.0` release notes for a full list of changes in the 5.3 series. +Security Fixes +============== + +- Fixed a security issue where any CrateDB user could read/import the content of + any file on the host system, the CrateDB process user has read access to, by + using the ``COPY FROM`` command with a file URI. This access is now restricted + to the ``crate`` superuser only. + Fixes =====
docs/sql/statements/copy-from.rst+2 −0 modified@@ -207,6 +207,8 @@ For example: The files must be accessible on at least one node and the system user running the ``crate`` process must have read access to every file specified. +Additionally, only the ``crate`` superuser is allowed to use the ``file://`` +scheme. By default, every node will attempt to import every file. If the file is accessible on multiple nodes, you can set the `shared`_ option to true in order
plugins/cr8-copy-s3/src/test/java/io/crate/copy/s3/S3FileReadingCollectorTest.java+1 −1 modified@@ -212,7 +212,7 @@ private BatchIterator<Row> createBatchIterator(Collection<String> fileUris, inputs.add(sourceUriFailureInput); } return FileReadingIterator.newInstance( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), compression,
server/src/main/java/io/crate/exceptions/UnauthorizedException.java+9 −1 modified@@ -21,10 +21,18 @@ package io.crate.exceptions; -public class UnauthorizedException extends RuntimeException implements UnscopedException { +import java.io.IOException; + +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.common.io.stream.StreamInput; + +public class UnauthorizedException extends ElasticsearchException implements UnscopedException { public UnauthorizedException(String message) { super(message); } + public UnauthorizedException(StreamInput in) throws IOException { + super(in); + } }
server/src/main/java/io/crate/execution/engine/collect/files/FileReadingIterator.java+3 −4 modified@@ -62,7 +62,7 @@ public class FileReadingIterator implements BatchIterator<Row> { @VisibleForTesting static final int MAX_SOCKET_TIMEOUT_RETRIES = 5; - public static BatchIterator<Row> newInstance(Collection<String> fileUris, + public static BatchIterator<Row> newInstance(Collection<URI> fileUris, List<Input<?>> inputs, Iterable<LineCollectorExpression<?>> collectorExpressions, String compression, @@ -114,7 +114,7 @@ public static BatchIterator<Row> newInstance(Collection<String> fileUris, private LineProcessor lineProcessor; @VisibleForTesting - FileReadingIterator(Collection<String> fileUris, + FileReadingIterator(Collection<URI> fileUris, List<? extends Input<?>> inputs, Iterable<LineCollectorExpression<?>> collectorExpressions, String compression, @@ -324,8 +324,7 @@ public static URI toURI(String fileUri) { } @Nullable - private FileInput toFileInput(String fileUri, Settings withClauseOptions) { - URI uri = toURI(fileUri); + private FileInput toFileInput(URI uri, Settings withClauseOptions) { FileInputFactory fileInputFactory = fileInputFactories.get(uri.getScheme()); if (fileInputFactory != null) { try {
server/src/main/java/io/crate/execution/engine/collect/sources/FileCollectSource.java+32 −12 modified@@ -21,11 +21,26 @@ package io.crate.execution.engine.collect.sources; +import static java.util.Objects.requireNonNull; + +import java.net.URI; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.concurrent.CompletableFuture; + +import org.elasticsearch.cluster.service.ClusterService; +import org.elasticsearch.common.inject.Inject; +import org.elasticsearch.common.inject.Singleton; + import io.crate.analyze.AnalyzedCopyFrom; import io.crate.analyze.SymbolEvaluator; import io.crate.common.annotations.VisibleForTesting; import io.crate.data.BatchIterator; import io.crate.data.Row; +import io.crate.exceptions.UnauthorizedException; import io.crate.execution.dsl.phases.CollectPhase; import io.crate.execution.dsl.phases.FileUriCollectPhase; import io.crate.execution.engine.collect.CollectTask; @@ -40,16 +55,7 @@ import io.crate.planner.operators.SubQueryResults; import io.crate.types.ArrayType; import io.crate.types.DataTypes; -import org.elasticsearch.cluster.service.ClusterService; -import org.elasticsearch.common.inject.Inject; -import org.elasticsearch.common.inject.Singleton; - -import java.util.Arrays; -import java.util.Collection; -import java.util.Collections; -import java.util.List; -import java.util.Map; -import java.util.concurrent.CompletableFuture; +import io.crate.user.UserLookup; @Singleton public class FileCollectSource implements CollectSource { @@ -58,13 +64,18 @@ public class FileCollectSource implements CollectSource { private final Map<String, FileInputFactory> fileInputFactoryMap; private final InputFactory inputFactory; private final NodeContext nodeCtx; + private final UserLookup userLookup; @Inject - public FileCollectSource(NodeContext nodeCtx, ClusterService clusterService, Map<String, FileInputFactory> fileInputFactoryMap) { + public FileCollectSource(NodeContext nodeCtx, + ClusterService clusterService, + Map<String, FileInputFactory> fileInputFactoryMap, + UserLookup userLookup) { this.fileInputFactoryMap = fileInputFactoryMap; this.nodeCtx = nodeCtx; this.inputFactory = new InputFactory(nodeCtx); this.clusterService = clusterService; + this.userLookup = userLookup; } @Override @@ -77,7 +88,16 @@ public CompletableFuture<BatchIterator<Row>> getIterator(TransactionContext txnC inputFactory.ctxForRefs(txnCtx, FileLineReferenceResolver::getImplementation); ctx.add(collectPhase.toCollect()); - List<String> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()); + var user = requireNonNull(userLookup.findUser(txnCtx.sessionSettings().userName()), "User who invoked a statement must exist"); + List<URI> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()).stream() + .map(s -> { + var uri = FileReadingIterator.toURI(s); + if (uri.getScheme().equals("file") && user.isSuperUser() == false) { + throw new UnauthorizedException("Only a superuser can read from the local file system"); + } + return uri; + }) + .toList(); return CompletableFuture.completedFuture(FileReadingIterator.newInstance( fileUris, ctx.topLevelInputs(),
server/src/main/java/org/elasticsearch/ElasticsearchException.java+6 −1 modified@@ -969,7 +969,12 @@ private enum ElasticsearchExceptionHandle { org.elasticsearch.cluster.coordination.NodeHealthCheckFailureException.class, org.elasticsearch.cluster.coordination.NodeHealthCheckFailureException::new, 175, - Version.V_5_2_0); + Version.V_5_2_0), + UNAUTHORIZED_EXCEPTION( + io.crate.exceptions.UnauthorizedException.class, + io.crate.exceptions.UnauthorizedException::new, + 177, + Version.V_5_3_9); final Class<? extends ElasticsearchException> exceptionClass; final CheckedFunction<StreamInput, ? extends ElasticsearchException, IOException> constructor;
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingCollectorTest.java+1 −1 modified@@ -238,7 +238,7 @@ private BatchIterator<Row> createBatchIterator(Collection<String> fileUris, inputs.add(sourceUriFailureInput); } return FileReadingIterator.newInstance( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), compression,
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingIteratorTest.java+4 −4 modified@@ -170,7 +170,7 @@ public void test_iterator_closes_current_reader_on_io_error() throws Exception { Supplier<BatchIterator<Row>> batchIteratorSupplier = () -> new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), null, @@ -228,7 +228,7 @@ public void test_consecutive_retries_will_not_result_in_duplicate_reads() throws Supplier<BatchIterator<Row>> batchIteratorSupplier = () -> new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), null, @@ -289,7 +289,7 @@ public void test_skipping_csv_headers_and_rows_combined_with_retry_logic() throw Supplier<BatchIterator<Row>> batchIteratorSupplier = () -> new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), null, @@ -340,7 +340,7 @@ private BatchIterator<Row> createBatchIterator(Collection<String> fileUris, List<Input<?>> inputs = Collections.singletonList(ctx.add(raw)); return FileReadingIterator.newInstance( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), null,
server/src/test/java/io/crate/execution/engine/collect/MapSideDataCollectOperationTest.java+7 −1 modified@@ -56,6 +56,7 @@ import io.crate.test.integration.CrateDummyClusterServiceUnitTest; import io.crate.testing.TestingRowConsumer; import io.crate.types.DataTypes; +import io.crate.user.User; public class MapSideDataCollectOperationTest extends CrateDummyClusterServiceUnitTest { @@ -65,7 +66,12 @@ public class MapSideDataCollectOperationTest extends CrateDummyClusterServiceUni @Test public void testFileUriCollect() throws Exception { - FileCollectSource fileCollectSource = new FileCollectSource(createNodeContext(), clusterService, Collections.emptyMap()); + FileCollectSource fileCollectSource = new FileCollectSource( + createNodeContext(), + clusterService, + Collections.emptyMap(), + () -> List.of(User.CRATE_USER) + ); File tmpFile = temporaryFolder.newFile("fileUriCollectOperation.json"); try (OutputStreamWriter writer = new OutputStreamWriter(new FileOutputStream(tmpFile), StandardCharsets.UTF_8)) {
server/src/test/java/io/crate/integrationtests/CopyIntegrationTest.java+23 −1 modified@@ -26,7 +26,7 @@ import static io.crate.testing.Asserts.assertThat; import static io.crate.testing.TestingHelpers.printedTable; import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST; -import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; import static org.hamcrest.Matchers.both; import static org.hamcrest.Matchers.contains; import static org.hamcrest.Matchers.containsString; @@ -68,9 +68,13 @@ import com.carrotsearch.randomizedtesting.LifecycleScope; +import io.crate.action.sql.Sessions; +import io.crate.exceptions.UnauthorizedException; import io.crate.testing.Asserts; import io.crate.testing.SQLResponse; import io.crate.testing.UseJdbc; +import io.crate.testing.UseRandomizedSchema; +import io.crate.user.UserLookup; @IntegTestCase.ClusterScope(numDataNodes = 2) public class CopyIntegrationTest extends SQLHttpIntegrationTest { @@ -1187,4 +1191,22 @@ public void test_copy_preserves_the_implied_sub_column_order() throws IOExceptio ")" ); } + + @UseRandomizedSchema(random = false) + @Test + public void test_copy_from_local_file_is_only_allowed_for_superusers() { + execute("CREATE TABLE quotes (id INT PRIMARY KEY, " + + "quote STRING INDEX USING FULLTEXT) WITH (number_of_replicas = 0)"); + execute("CREATE USER test_user"); + execute("GRANT ALL TO test_user"); + + var roles = cluster().getInstance(UserLookup.class); + var user = roles.findUser("test_user"); + var sqlOperations = cluster().getInstance(Sessions.class); + try (var session = sqlOperations.createSession(null, user)) { + assertThatThrownBy(() -> execute("COPY quotes FROM ?", new Object[]{copyFilePath + "test_copy_from.json"}, session)) + .isExactlyInstanceOf(UnauthorizedException.class) + .hasMessage("Only a superuser can read from the local file system"); + } + } }
b75aeeabf90fRestrict `COPY FROM` using local files to superuser
12 files changed · +85 −14
docs/appendices/release-notes/5.5.4.rst+8 −0 modified@@ -44,6 +44,14 @@ Version 5.5.4 - Unreleased See the :ref:`version_5.5.0` release notes for a full list of changes in the 5.5 series. +Security Fixes +============== + +- Fixed a security issue where any CrateDB user could read/import the content of + any file on the host system, the CrateDB process user has read access to, by + using the ``COPY FROM`` command with a file URI. This access is now restricted + to the ``crate`` superuser only. + Fixes =====
docs/sql/statements/copy-from.rst+2 −0 modified@@ -207,6 +207,8 @@ For example: The files must be accessible on at least one node and the system user running the ``crate`` process must have read access to every file specified. +Additionally, only the ``crate`` superuser is allowed to use the ``file://`` +scheme. By default, every node will attempt to import every file. If the file is accessible on multiple nodes, you can set the `shared`_ option to true in order
plugins/cr8-copy-s3/src/test/java/io/crate/copy/s3/S3FileReadingCollectorTest.java+1 −1 modified@@ -106,7 +106,7 @@ public void testCollectWithOneSocketTimeout() throws Throwable { private FileReadingIterator createBatchIterator(S3ObjectInputStream inputStream, String ... fileUris) { String compression = null; return new FileReadingIterator( - Arrays.asList(fileUris), + Arrays.stream(fileUris).map(FileReadingIterator::toURI).toList(), compression, Map.of( S3FileInputFactory.NAME,
server/src/main/java/io/crate/exceptions/UnauthorizedException.java+9 −1 modified@@ -21,10 +21,18 @@ package io.crate.exceptions; -public class UnauthorizedException extends RuntimeException implements UnscopedException { +import java.io.IOException; + +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.common.io.stream.StreamInput; + +public class UnauthorizedException extends ElasticsearchException implements UnscopedException { public UnauthorizedException(String message) { super(message); } + public UnauthorizedException(StreamInput in) throws IOException { + super(in); + } }
server/src/main/java/io/crate/execution/engine/collect/files/FileReadingIterator.java+2 −3 modified@@ -175,7 +175,7 @@ public boolean equals(Object obj) { } } - public FileReadingIterator(Collection<String> fileUris, + public FileReadingIterator(Collection<URI> fileUris, String compression, Map<String, FileInputFactory> fileInputFactories, Boolean shared, @@ -398,8 +398,7 @@ public static URI toURI(String fileUri) { } @Nullable - private FileInput toFileInput(String fileUri, Settings withClauseOptions) { - URI uri = toURI(fileUri); + private FileInput toFileInput(URI uri, Settings withClauseOptions) { FileInputFactory fileInputFactory = fileInputFactories.get(uri.getScheme()); if (fileInputFactory != null) { try {
server/src/main/java/io/crate/execution/engine/collect/sources/FileCollectSource.java+19 −2 modified@@ -21,6 +21,9 @@ package io.crate.execution.engine.collect.sources; +import static java.util.Objects.requireNonNull; + +import java.net.URI; import java.util.Arrays; import java.util.Collection; import java.util.Collections; @@ -40,6 +43,7 @@ import io.crate.data.BatchIterator; import io.crate.data.Row; import io.crate.data.SkippingBatchIterator; +import io.crate.exceptions.UnauthorizedException; import io.crate.execution.dsl.phases.CollectPhase; import io.crate.execution.dsl.phases.FileUriCollectPhase; import io.crate.execution.engine.collect.CollectTask; @@ -55,6 +59,7 @@ import io.crate.planner.operators.SubQueryResults; import io.crate.types.ArrayType; import io.crate.types.DataTypes; +import io.crate.user.UserLookup; @Singleton public class FileCollectSource implements CollectSource { @@ -64,17 +69,20 @@ public class FileCollectSource implements CollectSource { private final InputFactory inputFactory; private final NodeContext nodeCtx; private final ThreadPool threadPool; + private final UserLookup userLookup; @Inject public FileCollectSource(NodeContext nodeCtx, ClusterService clusterService, Map<String, FileInputFactory> fileInputFactoryMap, - ThreadPool threadPool) { + ThreadPool threadPool, + UserLookup userLookup) { this.fileInputFactoryMap = fileInputFactoryMap; this.nodeCtx = nodeCtx; this.inputFactory = new InputFactory(nodeCtx); this.clusterService = clusterService; this.threadPool = threadPool; + this.userLookup = userLookup; } @Override @@ -87,7 +95,16 @@ public CompletableFuture<BatchIterator<Row>> getIterator(TransactionContext txnC inputFactory.ctxForRefs(txnCtx, FileLineReferenceResolver::getImplementation); ctx.add(collectPhase.toCollect()); - List<String> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()); + var user = requireNonNull(userLookup.findUser(txnCtx.sessionSettings().userName()), "User who invoked a statement must exist"); + List<URI> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()).stream() + .map(s -> { + var uri = FileReadingIterator.toURI(s); + if (uri.getScheme().equals("file") && user.isSuperUser() == false) { + throw new UnauthorizedException("Only a superuser can read from the local file system"); + } + return uri; + }) + .toList(); FileReadingIterator fileReadingIterator = new FileReadingIterator( fileUris, fileUriCollectPhase.compression(),
server/src/main/java/org/elasticsearch/ElasticsearchException.java+6 −1 modified@@ -968,7 +968,12 @@ private enum ElasticsearchExceptionHandle { org.elasticsearch.cluster.coordination.NodeHealthCheckFailureException.class, org.elasticsearch.cluster.coordination.NodeHealthCheckFailureException::new, 175, - Version.V_5_2_0); + Version.V_5_2_0), + UNAUTHORIZED_EXCEPTION( + io.crate.exceptions.UnauthorizedException.class, + io.crate.exceptions.UnauthorizedException::new, + 177, + Version.V_5_5_4); final Class<? extends ElasticsearchException> exceptionClass; final CheckedFunction<StreamInput, ? extends ElasticsearchException, IOException> constructor;
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingCollectorTest.java+1 −1 modified@@ -200,7 +200,7 @@ private static FileReadingIterator it(String ... fileUris) { private static FileReadingIterator it(Collection<String> fileUris, String compression) { return new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), compression, Map.of(LocalFsFileInputFactory.NAME, new LocalFsFileInputFactory()), false,
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingIteratorTest.java+9 −3 modified@@ -34,6 +34,7 @@ import java.io.InputStream; import java.io.InputStreamReader; import java.net.SocketTimeoutException; +import java.net.URI; import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; @@ -43,6 +44,7 @@ import java.util.concurrent.ScheduledExecutorService; import java.util.concurrent.TimeUnit; import java.util.function.Supplier; +import java.util.stream.Stream; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.test.ESTestCase; @@ -87,7 +89,9 @@ public void test_iterator_closes_current_reader_on_io_error() throws Exception { Path tempFile2 = createTempFile("tempfile2", ".csv"); List<String> lines2 = List.of("name,id,age", "Trillian,5,33"); Files.write(tempFile2, lines2); - List<String> fileUris = List.of(tempFile1.toUri().toString(), tempFile2.toUri().toString()); + List<URI> fileUris = Stream.of(tempFile1.toUri().toString(), tempFile2.toUri().toString()) + .map(FileReadingIterator::toURI) + .toList(); Supplier<BatchIterator<LineCursor>> batchIteratorSupplier = () -> new FileReadingIterator( @@ -139,7 +143,8 @@ public void test_consecutive_retries_will_not_result_in_duplicate_reads() throws Path tempFile = createTempFile("tempfile1", ".csv"); List<String> lines = List.of("id", "1", "2", "3", "4", "5"); Files.write(tempFile, lines); - List<String> fileUris = List.of(tempFile.toUri().toString()); + List<URI> fileUris = Stream.of(tempFile.toUri().toString()) + .map(FileReadingIterator::toURI).toList(); Supplier<BatchIterator<LineCursor>> batchIteratorSupplier = () -> new FileReadingIterator( @@ -213,7 +218,8 @@ public void test_retry_from_one_uri_does_not_affect_reading_next_uri() throws Ex Files.write(tempFile, List.of("1", "2", "3")); Path tempFile2 = createTempFile("tempfile2", ".csv"); Files.write(tempFile2, List.of("4", "5", "6")); - List<String> fileUris = List.of(tempFile.toUri().toString(), tempFile2.toUri().toString()); + List<URI> fileUris = Stream.of(tempFile.toUri().toString(), tempFile2.toUri().toString()) + .map(FileReadingIterator::toURI).toList(); var fi = new FileReadingIterator( fileUris,
server/src/test/java/io/crate/execution/engine/collect/MapSideDataCollectOperationTest.java+3 −1 modified@@ -56,6 +56,7 @@ import io.crate.metadata.CoordinatorTxnCtx; import io.crate.test.integration.CrateDummyClusterServiceUnitTest; import io.crate.types.DataTypes; +import io.crate.user.User; public class MapSideDataCollectOperationTest extends CrateDummyClusterServiceUnitTest { @@ -69,7 +70,8 @@ public void testFileUriCollect() throws Exception { createNodeContext(), clusterService, Collections.emptyMap(), - THREAD_POOL + THREAD_POOL, + () -> List.of(User.CRATE_USER) ); File tmpFile = temporaryFolder.newFile("fileUriCollectOperation.json");
server/src/test/java/io/crate/execution/engine/collect/sources/FileCollectSourceTest.java+2 −1 modified@@ -92,7 +92,8 @@ public void test_file_collect_source_returns_iterator_that_can_skip_lines() thro new NodeContext(new Functions(Map.of()), userLookup), clusterService, Map.of(), - THREAD_POOL + THREAD_POOL, + () -> List.of(User.CRATE_USER) ); CompletableFuture<BatchIterator<Row>> iterator = fileCollectSource.getIterator(
server/src/test/java/io/crate/integrationtests/CopyIntegrationTest.java+23 −0 modified@@ -26,6 +26,7 @@ import static io.crate.testing.Asserts.assertThat; import static io.crate.testing.TestingHelpers.printedTable; import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST; +import static org.assertj.core.api.Assertions.assertThatThrownBy; import java.io.File; import java.io.FileOutputStream; @@ -53,10 +54,14 @@ import com.carrotsearch.randomizedtesting.LifecycleScope; +import io.crate.action.sql.Sessions; +import io.crate.exceptions.UnauthorizedException; import io.crate.testing.Asserts; import io.crate.testing.SQLResponse; import io.crate.testing.UseJdbc; import io.crate.testing.UseNewCluster; +import io.crate.testing.UseRandomizedSchema; +import io.crate.user.UserLookup; @IntegTestCase.ClusterScope(numDataNodes = 2) public class CopyIntegrationTest extends SQLHttpIntegrationTest { @@ -1196,4 +1201,22 @@ primary key (id) "2| 31123| apple safari| 23073" ); } + + @UseRandomizedSchema(random = false) + @Test + public void test_copy_from_local_file_is_only_allowed_for_superusers() { + execute("CREATE TABLE quotes (id INT PRIMARY KEY, " + + "quote STRING INDEX USING FULLTEXT) WITH (number_of_replicas = 0)"); + execute("CREATE USER test_user"); + execute("GRANT ALL TO test_user"); + + var roles = cluster().getInstance(UserLookup.class); + var user = roles.findUser("test_user"); + Sessions sqlOperations = cluster().getInstance(Sessions.class); + try (var session = sqlOperations.newSession(null, user)) { + assertThatThrownBy(() -> execute("COPY quotes FROM ?", new Object[]{copyFilePath + "test_copy_from.json"}, session)) + .isExactlyInstanceOf(UnauthorizedException.class) + .hasMessage("Only a superuser can read from the local file system"); + } + } }
32d0fc2ebb83Restrict `COPY FROM` using local files to superuser
13 files changed · +93 −29
benchmarks/src/main/java/io/crate/execution/engine/reader/CsvReaderBenchmark.java+1 −1 modified@@ -141,7 +141,7 @@ public void measureFileReadingIteratorForCSV(Blackhole blackhole) { List<Input<?>> inputs = Collections.singletonList(ctx.add(raw)); BatchIterator<Row> batchIterator = FileReadingIterator.newInstance( - Collections.singletonList(fileUri), + List.of(FileReadingIterator.toURI(fileUri)), inputs, ctx.expressions(), null,
benchmarks/src/main/java/io/crate/execution/engine/reader/JsonReaderBenchmark.java+1 −1 modified@@ -141,7 +141,7 @@ public void measureFileReadingIteratorForJson(Blackhole blackhole) { List<Input<?>> inputs = Collections.singletonList(ctx.add(raw)); BatchIterator<Row> batchIterator = FileReadingIterator.newInstance( - Collections.singletonList(fileUri), + List.of(FileReadingIterator.toURI(fileUri)), inputs, ctx.expressions(), null,
docs/appendices/release-notes/5.4.8.rst+7 −0 modified@@ -43,6 +43,13 @@ Version 5.4.8 - Unreleased See the :ref:`version_5.4.0` release notes for a full list of changes in the 5.4 series. +Security Fixes +============== + +- Fixed a security issue where any CrateDB user could read/import the content of + any file on the host system, the CrateDB process user has read access to, by + using the ``COPY FROM`` command with a file URI. This access is now restricted + to the ``crate`` superuser only. Fixes =====
docs/sql/statements/copy-from.rst+2 −0 modified@@ -207,6 +207,8 @@ For example: The files must be accessible on at least one node and the system user running the ``crate`` process must have read access to every file specified. +Additionally, only the ``crate`` superuser is allowed to use the ``file://`` +scheme. By default, every node will attempt to import every file. If the file is accessible on multiple nodes, you can set the `shared`_ option to true in order
plugins/cr8-copy-s3/src/test/java/io/crate/copy/s3/S3FileReadingCollectorTest.java+1 −1 modified@@ -217,7 +217,7 @@ private BatchIterator<Row> createBatchIterator(Collection<String> fileUris, inputs.add(sourceUriFailureInput); } return FileReadingIterator.newInstance( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), compression,
server/src/main/java/io/crate/exceptions/UnauthorizedException.java+9 −1 modified@@ -21,10 +21,18 @@ package io.crate.exceptions; -public class UnauthorizedException extends RuntimeException implements UnscopedException { +import java.io.IOException; + +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.common.io.stream.StreamInput; + +public class UnauthorizedException extends ElasticsearchException implements UnscopedException { public UnauthorizedException(String message) { super(message); } + public UnauthorizedException(StreamInput in) throws IOException { + super(in); + } }
server/src/main/java/io/crate/execution/engine/collect/files/FileReadingIterator.java+3 −4 modified@@ -67,7 +67,7 @@ public class FileReadingIterator implements BatchIterator<Row> { @VisibleForTesting static final int MAX_SOCKET_TIMEOUT_RETRIES = 5; - public static BatchIterator<Row> newInstance(Collection<String> fileUris, + public static BatchIterator<Row> newInstance(Collection<URI> fileUris, List<Input<?>> inputs, Iterable<LineCollectorExpression<?>> collectorExpressions, String compression, @@ -125,7 +125,7 @@ public static BatchIterator<Row> newInstance(Collection<String> fileUris, private final Iterator<TimeValue> backOffPolicy; @VisibleForTesting - FileReadingIterator(Collection<String> fileUris, + FileReadingIterator(Collection<URI> fileUris, List<? extends Input<?>> inputs, Iterable<LineCollectorExpression<?>> collectorExpressions, String compression, @@ -360,8 +360,7 @@ public static URI toURI(String fileUri) { } @Nullable - private FileInput toFileInput(String fileUri, Settings withClauseOptions) { - URI uri = toURI(fileUri); + private FileInput toFileInput(URI uri, Settings withClauseOptions) { FileInputFactory fileInputFactory = fileInputFactories.get(uri.getScheme()); if (fileInputFactory != null) { try {
server/src/main/java/io/crate/execution/engine/collect/sources/FileCollectSource.java+31 −13 modified@@ -21,11 +21,27 @@ package io.crate.execution.engine.collect.sources; +import static java.util.Objects.requireNonNull; + +import java.net.URI; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.concurrent.CompletableFuture; + +import org.elasticsearch.cluster.service.ClusterService; +import org.elasticsearch.common.inject.Inject; +import org.elasticsearch.common.inject.Singleton; +import org.elasticsearch.threadpool.ThreadPool; + import io.crate.analyze.AnalyzedCopyFrom; import io.crate.analyze.SymbolEvaluator; import io.crate.common.annotations.VisibleForTesting; import io.crate.data.BatchIterator; import io.crate.data.Row; +import io.crate.exceptions.UnauthorizedException; import io.crate.execution.dsl.phases.CollectPhase; import io.crate.execution.dsl.phases.FileUriCollectPhase; import io.crate.execution.engine.collect.CollectTask; @@ -40,17 +56,7 @@ import io.crate.planner.operators.SubQueryResults; import io.crate.types.ArrayType; import io.crate.types.DataTypes; -import org.elasticsearch.cluster.service.ClusterService; -import org.elasticsearch.common.inject.Inject; -import org.elasticsearch.common.inject.Singleton; -import org.elasticsearch.threadpool.ThreadPool; - -import java.util.Arrays; -import java.util.Collection; -import java.util.Collections; -import java.util.List; -import java.util.Map; -import java.util.concurrent.CompletableFuture; +import io.crate.user.UserLookup; @Singleton public class FileCollectSource implements CollectSource { @@ -60,17 +66,20 @@ public class FileCollectSource implements CollectSource { private final InputFactory inputFactory; private final NodeContext nodeCtx; private final ThreadPool threadPool; + private final UserLookup userLookup; @Inject public FileCollectSource(NodeContext nodeCtx, ClusterService clusterService, Map<String, FileInputFactory> fileInputFactoryMap, - ThreadPool threadPool) { + ThreadPool threadPool, + UserLookup userLookup) { this.fileInputFactoryMap = fileInputFactoryMap; this.nodeCtx = nodeCtx; this.inputFactory = new InputFactory(nodeCtx); this.clusterService = clusterService; this.threadPool = threadPool; + this.userLookup = userLookup; } @Override @@ -83,7 +92,16 @@ public CompletableFuture<BatchIterator<Row>> getIterator(TransactionContext txnC inputFactory.ctxForRefs(txnCtx, FileLineReferenceResolver::getImplementation); ctx.add(collectPhase.toCollect()); - List<String> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()); + var user = requireNonNull(userLookup.findUser(txnCtx.sessionSettings().userName()), "User who invoked a statement must exist"); + List<URI> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()).stream() + .map(s -> { + var uri = FileReadingIterator.toURI(s); + if (uri.getScheme().equals("file") && user.isSuperUser() == false) { + throw new UnauthorizedException("Only a superuser can read from the local file system"); + } + return uri; + }) + .toList(); return CompletableFuture.completedFuture( FileReadingIterator.newInstance( fileUris,
server/src/main/java/org/elasticsearch/ElasticsearchException.java+6 −1 modified@@ -969,7 +969,12 @@ private enum ElasticsearchExceptionHandle { org.elasticsearch.cluster.coordination.NodeHealthCheckFailureException.class, org.elasticsearch.cluster.coordination.NodeHealthCheckFailureException::new, 175, - Version.V_5_2_0); + Version.V_5_2_0), + UNAUTHORIZED_EXCEPTION( + io.crate.exceptions.UnauthorizedException.class, + io.crate.exceptions.UnauthorizedException::new, + 177, + Version.V_5_4_8); final Class<? extends ElasticsearchException> exceptionClass; final CheckedFunction<StreamInput, ? extends ElasticsearchException, IOException> constructor;
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingCollectorTest.java+1 −1 modified@@ -243,7 +243,7 @@ private BatchIterator<Row> createBatchIterator(Collection<String> fileUris, inputs.add(sourceUriFailureInput); } return FileReadingIterator.newInstance( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), compression,
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingIteratorTest.java+5 −5 modified@@ -196,7 +196,7 @@ public void test_iterator_closes_current_reader_on_io_error() throws Exception { Supplier<BatchIterator<Row>> batchIteratorSupplier = () -> new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), null, @@ -255,7 +255,7 @@ public void test_consecutive_retries_will_not_result_in_duplicate_reads() throws Supplier<BatchIterator<Row>> batchIteratorSupplier = () -> new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), null, @@ -317,7 +317,7 @@ public void test_skipping_csv_headers_and_rows_combined_with_retry_logic() throw Supplier<BatchIterator<Row>> batchIteratorSupplier = () -> new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), null, @@ -411,7 +411,7 @@ public void test_retry_from_one_uri_does_not_affect_reading_next_uri() throws Ex List<Input<?>> inputs = Collections.singletonList(ctx.add(raw)); var fi = new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), null, @@ -477,7 +477,7 @@ private BatchIterator<Row> createBatchIterator(Collection<String> fileUris, List<Input<?>> inputs = Collections.singletonList(ctx.add(raw)); return FileReadingIterator.newInstance( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), inputs, ctx.expressions(), null,
server/src/test/java/io/crate/execution/engine/collect/MapSideDataCollectOperationTest.java+3 −1 modified@@ -56,6 +56,7 @@ import io.crate.metadata.CoordinatorTxnCtx; import io.crate.test.integration.CrateDummyClusterServiceUnitTest; import io.crate.types.DataTypes; +import io.crate.user.User; public class MapSideDataCollectOperationTest extends CrateDummyClusterServiceUnitTest { @@ -69,7 +70,8 @@ public void testFileUriCollect() throws Exception { createNodeContext(), clusterService, Collections.emptyMap(), - THREAD_POOL + THREAD_POOL, + () -> List.of(User.CRATE_USER) ); File tmpFile = temporaryFolder.newFile("fileUriCollectOperation.json");
server/src/test/java/io/crate/integrationtests/CopyIntegrationTest.java+23 −0 modified@@ -26,6 +26,7 @@ import static io.crate.testing.Asserts.assertThat; import static io.crate.testing.TestingHelpers.printedTable; import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST; +import static org.assertj.core.api.Assertions.assertThatThrownBy; import static org.hamcrest.Matchers.both; import static org.hamcrest.Matchers.contains; import static org.hamcrest.Matchers.containsString; @@ -67,9 +68,13 @@ import com.carrotsearch.randomizedtesting.LifecycleScope; +import io.crate.action.sql.Sessions; +import io.crate.exceptions.UnauthorizedException; import io.crate.testing.Asserts; import io.crate.testing.SQLResponse; import io.crate.testing.UseJdbc; +import io.crate.testing.UseRandomizedSchema; +import io.crate.user.UserLookup; @IntegTestCase.ClusterScope(numDataNodes = 2) public class CopyIntegrationTest extends SQLHttpIntegrationTest { @@ -1217,4 +1222,22 @@ create table tbl (x int, created generated always as round((random() + 1) * 100) assertThat(response.rows()[0][1]).isEqualTo(created); } } + + @UseRandomizedSchema(random = false) + @Test + public void test_copy_from_local_file_is_only_allowed_for_superusers() { + execute("CREATE TABLE quotes (id INT PRIMARY KEY, " + + "quote STRING INDEX USING FULLTEXT) WITH (number_of_replicas = 0)"); + execute("CREATE USER test_user"); + execute("GRANT ALL TO test_user"); + + var roles = cluster().getInstance(UserLookup.class); + var user = roles.findUser("test_user"); + var sqlOperations = cluster().getInstance(Sessions.class); + try (var session = sqlOperations.createSession(null, user)) { + assertThatThrownBy(() -> execute("COPY quotes FROM ?", new Object[]{copyFilePath + "test_copy_from.json"}, session)) + .isExactlyInstanceOf(UnauthorizedException.class) + .hasMessage("Only a superuser can read from the local file system"); + } + } }
c5034323f1b5Restrict `COPY FROM` using local files to superuser
13 files changed · +94 −14
docs/appendices/release-notes/5.5.4.rst+8 −0 modified@@ -44,6 +44,14 @@ Version 5.5.4 - Unreleased See the :ref:`version_5.5.0` release notes for a full list of changes in the 5.5 series. +Security Fixes +============== + +- Fixed a security issue where any CrateDB user could read/import the content of + any file on the host system, the CrateDB process user has read access to, by + using the ``COPY FROM`` command with a file URI. This access is now restricted + to the ``crate`` superuser only. + Fixes =====
docs/appendices/release-notes/5.6.1.rst+7 −0 modified@@ -43,6 +43,13 @@ Version 5.6.1 - Unreleased See the :ref:`version_5.6.0` release notes for a full list of changes in the 5.6 series. +Security Fixes +============== + +- Fixed a security issue where any CrateDB user could read/import the content of + any file on the host system, the CrateDB process user has read access to, by + using the ``COPY FROM`` command with a file URI. This access is now restricted + to the ``crate`` superuser only. Fixes =====
docs/sql/statements/copy-from.rst+2 −0 modified@@ -211,6 +211,8 @@ For example: The files must be accessible on at least one node and the system user running the ``crate`` process must have read access to every file specified. +Additionally, only the ``crate`` superuser is allowed to use the ``file://`` +scheme. By default, every node will attempt to import every file. If the file is accessible on multiple nodes, you can set the `shared`_ option to true in order
plugins/cr8-copy-s3/src/test/java/io/crate/copy/s3/S3FileReadingCollectorTest.java+1 −1 modified@@ -106,7 +106,7 @@ public void testCollectWithOneSocketTimeout() throws Throwable { private FileReadingIterator createBatchIterator(S3ObjectInputStream inputStream, String ... fileUris) { String compression = null; return new FileReadingIterator( - Arrays.asList(fileUris), + Arrays.stream(fileUris).map(FileReadingIterator::toURI).toList(), compression, Map.of( S3FileInputFactory.NAME,
server/src/main/java/io/crate/exceptions/UnauthorizedException.java+9 −1 modified@@ -21,10 +21,18 @@ package io.crate.exceptions; -public class UnauthorizedException extends RuntimeException implements UnscopedException { +import java.io.IOException; + +import org.elasticsearch.ElasticsearchException; +import org.elasticsearch.common.io.stream.StreamInput; + +public class UnauthorizedException extends ElasticsearchException implements UnscopedException { public UnauthorizedException(String message) { super(message); } + public UnauthorizedException(StreamInput in) throws IOException { + super(in); + } }
server/src/main/java/io/crate/execution/engine/collect/files/FileReadingIterator.java+2 −3 modified@@ -175,7 +175,7 @@ public boolean equals(Object obj) { } } - public FileReadingIterator(Collection<String> fileUris, + public FileReadingIterator(Collection<URI> fileUris, String compression, Map<String, FileInputFactory> fileInputFactories, Boolean shared, @@ -398,8 +398,7 @@ public static URI toURI(String fileUri) { } @Nullable - private FileInput toFileInput(String fileUri, Settings withClauseOptions) { - URI uri = toURI(fileUri); + private FileInput toFileInput(URI uri, Settings withClauseOptions) { FileInputFactory fileInputFactory = fileInputFactories.get(uri.getScheme()); if (fileInputFactory != null) { try {
server/src/main/java/io/crate/execution/engine/collect/sources/FileCollectSource.java+20 −2 modified@@ -21,6 +21,9 @@ package io.crate.execution.engine.collect.sources; +import static java.util.Objects.requireNonNull; + +import java.net.URI; import java.util.Arrays; import java.util.Collection; import java.util.Collections; @@ -40,6 +43,7 @@ import io.crate.data.BatchIterator; import io.crate.data.Row; import io.crate.data.SkippingBatchIterator; +import io.crate.exceptions.UnauthorizedException; import io.crate.execution.dsl.phases.CollectPhase; import io.crate.execution.dsl.phases.FileUriCollectPhase; import io.crate.execution.engine.collect.CollectTask; @@ -53,6 +57,8 @@ import io.crate.metadata.NodeContext; import io.crate.metadata.TransactionContext; import io.crate.planner.operators.SubQueryResults; +import io.crate.role.Role; +import io.crate.role.Roles; import io.crate.types.DataTypes; @Singleton @@ -63,17 +69,20 @@ public class FileCollectSource implements CollectSource { private final InputFactory inputFactory; private final NodeContext nodeCtx; private final ThreadPool threadPool; + private final Roles roles; @Inject public FileCollectSource(NodeContext nodeCtx, ClusterService clusterService, Map<String, FileInputFactory> fileInputFactoryMap, - ThreadPool threadPool) { + ThreadPool threadPool, + Roles roles) { this.fileInputFactoryMap = fileInputFactoryMap; this.nodeCtx = nodeCtx; this.inputFactory = new InputFactory(nodeCtx); this.clusterService = clusterService; this.threadPool = threadPool; + this.roles = roles; } @Override @@ -86,7 +95,16 @@ public CompletableFuture<BatchIterator<Row>> getIterator(TransactionContext txnC inputFactory.ctxForRefs(txnCtx, FileLineReferenceResolver::getImplementation); ctx.add(collectPhase.toCollect()); - List<String> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()); + Role user = requireNonNull(roles.findUser(txnCtx.sessionSettings().userName()), "User who invoked a statement must exist"); + List<URI> fileUris = targetUriToStringList(txnCtx, nodeCtx, fileUriCollectPhase.targetUri()).stream() + .map(s -> { + var uri = FileReadingIterator.toURI(s); + if (uri.getScheme().equals("file") && user.isSuperUser() == false) { + throw new UnauthorizedException("Only a superuser can read from the local file system"); + } + return uri; + }) + .toList(); FileReadingIterator fileReadingIterator = new FileReadingIterator( fileUris, fileUriCollectPhase.compression(),
server/src/main/java/org/elasticsearch/ElasticsearchException.java+6 −1 modified@@ -972,7 +972,12 @@ private enum ElasticsearchExceptionHandle { io.crate.exceptions.OperationOnInaccessibleRelationException.class, io.crate.exceptions.OperationOnInaccessibleRelationException::new, 176, - Version.V_5_6_0); + Version.V_5_6_0), + UNAUTHORIZED_EXCEPTION( + io.crate.exceptions.UnauthorizedException.class, + io.crate.exceptions.UnauthorizedException::new, + 177, + Version.V_5_6_1); final Class<? extends ElasticsearchException> exceptionClass; final CheckedFunction<StreamInput, ? extends ElasticsearchException, IOException> constructor;
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingCollectorTest.java+1 −1 modified@@ -200,7 +200,7 @@ private static FileReadingIterator it(String ... fileUris) { private static FileReadingIterator it(Collection<String> fileUris, String compression) { return new FileReadingIterator( - fileUris, + fileUris.stream().map(FileReadingIterator::toURI).toList(), compression, Map.of(LocalFsFileInputFactory.NAME, new LocalFsFileInputFactory()), false,
server/src/test/java/io/crate/execution/engine/collect/files/FileReadingIteratorTest.java+9 −3 modified@@ -34,6 +34,7 @@ import java.io.InputStream; import java.io.InputStreamReader; import java.net.SocketTimeoutException; +import java.net.URI; import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; @@ -43,6 +44,7 @@ import java.util.concurrent.ScheduledExecutorService; import java.util.concurrent.TimeUnit; import java.util.function.Supplier; +import java.util.stream.Stream; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.test.ESTestCase; @@ -87,7 +89,9 @@ public void test_iterator_closes_current_reader_on_io_error() throws Exception { Path tempFile2 = createTempFile("tempfile2", ".csv"); List<String> lines2 = List.of("name,id,age", "Trillian,5,33"); Files.write(tempFile2, lines2); - List<String> fileUris = List.of(tempFile1.toUri().toString(), tempFile2.toUri().toString()); + List<URI> fileUris = Stream.of(tempFile1.toUri().toString(), tempFile2.toUri().toString()) + .map(FileReadingIterator::toURI) + .toList(); Supplier<BatchIterator<LineCursor>> batchIteratorSupplier = () -> new FileReadingIterator( @@ -139,7 +143,8 @@ public void test_consecutive_retries_will_not_result_in_duplicate_reads() throws Path tempFile = createTempFile("tempfile1", ".csv"); List<String> lines = List.of("id", "1", "2", "3", "4", "5"); Files.write(tempFile, lines); - List<String> fileUris = List.of(tempFile.toUri().toString()); + List<URI> fileUris = Stream.of(tempFile.toUri().toString()) + .map(FileReadingIterator::toURI).toList(); Supplier<BatchIterator<LineCursor>> batchIteratorSupplier = () -> new FileReadingIterator( @@ -213,7 +218,8 @@ public void test_retry_from_one_uri_does_not_affect_reading_next_uri() throws Ex Files.write(tempFile, List.of("1", "2", "3")); Path tempFile2 = createTempFile("tempfile2", ".csv"); Files.write(tempFile2, List.of("4", "5", "6")); - List<String> fileUris = List.of(tempFile.toUri().toString(), tempFile2.toUri().toString()); + List<URI> fileUris = Stream.of(tempFile.toUri().toString(), tempFile2.toUri().toString()) + .map(FileReadingIterator::toURI).toList(); var fi = new FileReadingIterator( fileUris,
server/src/test/java/io/crate/execution/engine/collect/MapSideDataCollectOperationTest.java+3 −1 modified@@ -54,6 +54,7 @@ import io.crate.expression.symbol.Literal; import io.crate.metadata.ColumnIdent; import io.crate.metadata.CoordinatorTxnCtx; +import io.crate.role.Role; import io.crate.test.integration.CrateDummyClusterServiceUnitTest; import io.crate.types.DataTypes; @@ -69,7 +70,8 @@ public void testFileUriCollect() throws Exception { createNodeContext(), clusterService, Collections.emptyMap(), - THREAD_POOL + THREAD_POOL, + () -> List.of(Role.CRATE_USER) ); File tmpFile = temporaryFolder.newFile("fileUriCollectOperation.json");
server/src/test/java/io/crate/execution/engine/collect/sources/FileCollectSourceTest.java+2 −1 modified@@ -92,7 +92,8 @@ public void test_file_collect_source_returns_iterator_that_can_skip_lines() thro new NodeContext(new Functions(Map.of()), roles), clusterService, Map.of(), - THREAD_POOL + THREAD_POOL, + () -> List.of(Role.CRATE_USER) ); CompletableFuture<BatchIterator<Row>> iterator = fileCollectSource.getIterator(
server/src/test/java/io/crate/integrationtests/CopyIntegrationTest.java+24 −0 modified@@ -26,6 +26,7 @@ import static io.crate.testing.Asserts.assertThat; import static io.crate.testing.TestingHelpers.printedTable; import static io.netty.handler.codec.http.HttpResponseStatus.BAD_REQUEST; +import static org.assertj.core.api.Assertions.assertThatThrownBy; import java.io.File; import java.io.FileOutputStream; @@ -53,10 +54,15 @@ import com.carrotsearch.randomizedtesting.LifecycleScope; +import io.crate.action.sql.Sessions; +import io.crate.exceptions.UnauthorizedException; +import io.crate.role.Role; +import io.crate.role.Roles; import io.crate.testing.Asserts; import io.crate.testing.SQLResponse; import io.crate.testing.UseJdbc; import io.crate.testing.UseNewCluster; +import io.crate.testing.UseRandomizedSchema; @IntegTestCase.ClusterScope(numDataNodes = 2) public class CopyIntegrationTest extends SQLHttpIntegrationTest { @@ -1196,4 +1202,22 @@ primary key (id) "2| 31123| apple safari| 23073" ); } + + @UseRandomizedSchema(random = false) + @Test + public void test_copy_from_local_file_is_only_allowed_for_superusers() { + execute("CREATE TABLE quotes (id INT PRIMARY KEY, " + + "quote STRING INDEX USING FULLTEXT) WITH (number_of_replicas = 0)"); + execute("CREATE USER test_user"); + execute("GRANT ALL TO test_user"); + + var roles = cluster().getInstance(Roles.class); + Role user = roles.findUser("test_user"); + Sessions sqlOperations = cluster().getInstance(Sessions.class); + try (var session = sqlOperations.newSession(null, user)) { + assertThatThrownBy(() -> execute("COPY quotes FROM ?", new Object[]{copyFilePath + "test_copy_from.json"}, session)) + .isExactlyInstanceOf(UnauthorizedException.class) + .hasMessage("Only a superuser can read from the local file system"); + } + } }
Vulnerability mechanics
Generated on May 9, 2026. Inputs: CWE entries + fix-commit diffs from this CVE's patches. Citations validated against bundle.
References
8- github.com/advisories/GHSA-475g-vj6c-xf96ghsaADVISORY
- nvd.nist.gov/vuln/detail/CVE-2024-24565ghsaADVISORY
- github.com/crate/crate/commit/32d0fc2ebb834ea324eb7ab5d01320a67bc5c3c7ghsaWEB
- github.com/crate/crate/commit/4e857d675683095945dd524d6ba03e692c70ecd6ghsax_refsource_MISCWEB
- github.com/crate/crate/commit/b75aeeabf90f51bd96ddb499903928fd10185207ghsaWEB
- github.com/crate/crate/commit/c4c97d5a1c52cc2250ea42d062a3d37550c19dd5ghsaWEB
- github.com/crate/crate/commit/c5034323f1b56ca5d04b8ef4c6029eb63a5ba172ghsaWEB
- github.com/crate/crate/security/advisories/GHSA-475g-vj6c-xf96ghsax_refsource_CONFIRMWEB
News mentions
0No linked articles in our index yet.