CVE-2018-8026
Description
This vulnerability in Apache Solr 6.0.0 to 6.6.4 and 7.0.0 to 7.3.1 relates to an XML external entity expansion (XXE) in Solr config files (currency.xml, enumsConfig.xml referred from schema.xml, TIKA parsecontext config file). In addition, Xinclude functionality provided in these config files is also affected in a similar way. The vulnerability can be used as XXE using file/ftp/http protocols in order to read arbitrary local files from the Solr server or the internal network. The manipulated files can be uploaded as configsets using Solr's API, allowing to exploit that vulnerability.
AI Insight
LLM-synthesized narrative grounded in this CVE's description and references.
An XXE vulnerability in Apache Solr's XML config file parsing allows remote attackers to read arbitrary files via crafted configsets uploaded through the API.
Vulnerability
Apache Solr versions 6.0.0 to 6.6.4 and 7.0.0 to 7.3.1 contain an XML external entity expansion (XXE) vulnerability in the parsing of configuration files, specifically currency.xml, enumsConfig.xml (referenced from schema.xml), and the TIKA parse context config file [1][2]. The DocumentBuilderFactory in FileExchangeRateProvider and AbstractEnumField does not disable external entity processing, allowing attackers to leverage file, FTP, or HTTP protocols for entity resolution [2].
Exploitation
An attacker with network access to the Solr API can upload a malicious configset containing a crafted XML file (e.g., a currency.xml with an XXE payload) and trigger the vulnerability [1][2]. The attacker does not require authentication if the Solr API is exposed; the vulnerability can be exploited by sending a specially crafted request to the configset upload endpoint [2]. No user interaction is needed beyond the initial upload.
Impact
Successful exploitation allows the attacker to read arbitrary local files from the Solr server’s filesystem or internal network resources via file retrieval and server-side request forgery [1][2]. This can lead to disclosure of sensitive configuration data, credentials, or other system information, potentially facilitating further compromise.
Mitigation
The vulnerability is fixed in Apache Solr versions 6.6.5 and 7.3.2, released on 2018-08-01 [3][4]. The fix replaces the unsafe XML parsing with SafeXMLParsing from org.apache.solr.util, which disables external entity resolution and XInclude processing [3][4]. Administrators should upgrade immediately; no workaround is available for unpatched versions. The vulnerability is not listed on the CISA KEV at this time.
AI Insight generated on May 22, 2026. Synthesized from this CVE's description and the cited reference URLs; citations are validated against the source bundle.
Affected packages
Versions sourced from the GitHub Security Advisory.
| Package | Affected versions | Patched versions |
|---|---|---|
org.apache.solr:solr-coreMaven | >= 7.0.0, < 7.4.0 | 7.4.0 |
org.apache.solr:solr-coreMaven | >= 6.0.0, < 6.6.5 | 6.6.5 |
Affected products
2- Apache Software Foundation/Apache Solrv5Range: 6.0.0 to 6.6.4
Patches
81880d4824e6cSOLR-12450: Don't allow referal to external resources in various config files
5 files changed · +287 −69
solr/CHANGES.txt+19 −0 modified@@ -221,6 +221,25 @@ Other Changes * SOLR-11122: Creating a core should write a core.properties file first and clean up on failure (Erick Erickson) +================== 6.6.5 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.14.v20161028 + +Bug Fixes +---------------------- + + * SOLR-12450: Don't allow referal to external resources in various config files. + (Yuyang Xiao, Uwe Schindler) + ================== 6.6.4 ================== Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
solr/contrib/extraction/src/java/org/apache/solr/handler/extraction/ParseContextConfig.java+7 −5 modified@@ -16,27 +16,31 @@ */ package org.apache.solr.handler.extraction; -import javax.xml.parsers.DocumentBuilderFactory; import java.beans.BeanInfo; import java.beans.Introspector; import java.beans.PropertyDescriptor; import java.beans.PropertyEditor; import java.beans.PropertyEditorManager; -import java.io.InputStream; +import java.lang.invoke.MethodHandles; import java.lang.reflect.Method; import java.util.HashMap; import java.util.Locale; import java.util.Map; import org.apache.solr.core.SolrResourceLoader; +import org.apache.solr.util.SafeXMLParsing; import org.apache.tika.parser.ParseContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.w3c.dom.Document; import org.w3c.dom.Element; import org.w3c.dom.NamedNodeMap; import org.w3c.dom.Node; import org.w3c.dom.NodeList; public class ParseContextConfig { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + private final Map<Class<?>, Object> entries = new HashMap<>(); /** Creates an empty Config without any settings (used as placeholder). */ @@ -54,9 +58,7 @@ public ParseContextConfig(SolrResourceLoader resourceLoader, String parseContext } private static Document loadConfigFile(SolrResourceLoader resourceLoader, String parseContextConfigLoc) throws Exception { - try (InputStream in = resourceLoader.openResource(parseContextConfigLoc)) { - return DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(in, parseContextConfigLoc); - } + return SafeXMLParsing.parseConfigXML(log, resourceLoader, parseContextConfigLoc); } private void extract(Element element, SolrResourceLoader loader) throws Exception {
solr/core/src/java/org/apache/solr/schema/FileExchangeRateProvider.java+42 −64 modified@@ -17,22 +17,21 @@ package org.apache.solr.schema; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; -import javax.xml.xpath.XPath; -import javax.xml.xpath.XPathConstants; -import javax.xml.xpath.XPathExpressionException; -import javax.xml.xpath.XPathFactory; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Set; +import javax.xml.xpath.XPath; +import javax.xml.xpath.XPathConstants; +import javax.xml.xpath.XPathExpressionException; +import javax.xml.xpath.XPathFactory; + import org.apache.lucene.analysis.util.ResourceLoader; import org.apache.solr.common.SolrException; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -46,6 +45,7 @@ */ class FileExchangeRateProvider implements ExchangeRateProvider { private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + protected static final String PARAM_CURRENCY_CONFIG = "currencyConfig"; // Exchange rate map, maps Currency Code -> Currency Code -> Rate @@ -159,71 +159,49 @@ public Set<String> listAvailableCurrencies() { @Override public boolean reload() throws SolrException { - InputStream is = null; Map<String, Map<String, Double>> tmpRates = new HashMap<>(); + log.debug("Reloading exchange rates from file {}", currencyConfigFile); + try { - log.debug("Reloading exchange rates from file "+this.currencyConfigFile); - - is = loader.openResource(currencyConfigFile); - javax.xml.parsers.DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - dbf.setXIncludeAware(true); - dbf.setNamespaceAware(true); - } catch (UnsupportedOperationException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); - } + Document doc = SafeXMLParsing.parseConfigXML(log, loader, currencyConfigFile); + XPathFactory xpathFactory = XPathFactory.newInstance(); + XPath xpath = xpathFactory.newXPath(); + + // Parse exchange rates. + NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); - try { - Document doc = dbf.newDocumentBuilder().parse(is); - XPathFactory xpathFactory = XPathFactory.newInstance(); - XPath xpath = xpathFactory.newXPath(); + for (int i = 0; i < nodes.getLength(); i++) { + Node rateNode = nodes.item(i); + NamedNodeMap attributes = rateNode.getAttributes(); + Node from = attributes.getNamedItem("from"); + Node to = attributes.getNamedItem("to"); + Node rate = attributes.getNamedItem("rate"); + + if (from == null || to == null || rate == null) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); + } - // Parse exchange rates. - NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); + String fromCurrency = from.getNodeValue(); + String toCurrency = to.getNodeValue(); + Double exchangeRate; - for (int i = 0; i < nodes.getLength(); i++) { - Node rateNode = nodes.item(i); - NamedNodeMap attributes = rateNode.getAttributes(); - Node from = attributes.getNamedItem("from"); - Node to = attributes.getNamedItem("to"); - Node rate = attributes.getNamedItem("rate"); - - if (from == null || to == null || rate == null) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); - } - - String fromCurrency = from.getNodeValue(); - String toCurrency = to.getNodeValue(); - Double exchangeRate; - - if (null == CurrencyFieldType.getCurrency(fromCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); - } - if (null == CurrencyFieldType.getCurrency(toCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); - } - - try { - exchangeRate = Double.parseDouble(rate.getNodeValue()); - } catch (NumberFormatException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); - } - - addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); + if (null == CurrencyFieldType.getCurrency(fromCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); } - } catch (SAXException | XPathExpressionException | ParserConfigurationException | IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error parsing currency config.", e); - } - } catch (IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while opening Currency configuration file "+currencyConfigFile, e); - } finally { - try { - if (is != null) { - is.close(); + if (null == CurrencyFieldType.getCurrency(toCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); + } + + try { + exchangeRate = Double.parseDouble(rate.getNodeValue()); + } catch (NumberFormatException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); } - } catch (IOException e) { - e.printStackTrace(); + + addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); } + } catch (SAXException | IOException | XPathExpressionException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while parsing currency configuration file "+currencyConfigFile, e); } // Atomically swap in the new rates map, if it loaded successfully this.rates = tmpRates;
solr/core/src/java/org/apache/solr/util/SafeXMLParsing.java+120 −0 added@@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.FilterReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.io.StringReader; + +import javax.xml.XMLConstants; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.parsers.ParserConfigurationException; + +import org.apache.commons.io.input.CloseShieldInputStream; +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.solr.common.EmptyEntityResolver; +import org.apache.solr.common.SolrException; +import org.apache.solr.common.util.SuppressForbidden; +import org.apache.solr.common.util.XMLErrorLogger; +import org.slf4j.Logger; +import org.w3c.dom.Document; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; + +/** + * Some utility methods for parsing XML in a safe way. This class can be used to parse XML + * coming from network (completely untrusted) or it can load a config file from a + * {@link ResourceLoader}. In this case it allows external entities and xincludes, but only + * referring to files reachable by the loader. + */ +@SuppressForbidden(reason = "This class uses XML APIs directly that should not be used anywhere else in Solr code") +public final class SafeXMLParsing { + + public static final String SYSTEMID_UNTRUSTED = "untrusted://stream"; + + private SafeXMLParsing() {} + + /** Parses a config file from ResourceLoader. Xinclude and external entities are enabled, but cannot escape the resource loader. */ + public static Document parseConfigXML(Logger log, ResourceLoader loader, String file) throws SAXException, IOException { + try (InputStream in = loader.openResource(file)) { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + try { + dbf.setXIncludeAware(true); + } catch (UnsupportedOperationException e) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); + } + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(new SystemIdResolver(loader)); + db.setErrorHandler(new XMLErrorLogger(log)); + return db.parse(in, SystemIdResolver.createSystemIdFromResourceName(file)); + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given InputStream is not closed. */ + public static Document parseUntrustedXML(Logger log, InputStream in) throws SAXException, IOException { + return getUntrustedDocumentBuilder(log).parse(new CloseShieldInputStream(in), SYSTEMID_UNTRUSTED); + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given Reader is not closed. */ + public static Document parseUntrustedXML(Logger log, Reader reader) throws SAXException, IOException { + final InputSource is = new InputSource(new FilterReader(reader) { + @Override public void close() {} + }); + is.setSystemId(SYSTEMID_UNTRUSTED); + return getUntrustedDocumentBuilder(log).parse(is); + } + + public static Document parseUntrustedXML(Logger log, String xml) throws SAXException, IOException { + return parseUntrustedXML(log, new StringReader(xml)); + } + + private static DocumentBuilder getUntrustedDocumentBuilder(Logger log) { + try { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE); + db.setErrorHandler(new XMLErrorLogger(log)); + return db; + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + private static void trySetDOMFeature(DocumentBuilderFactory factory, String feature, boolean enabled) { + try { + factory.setFeature(feature, enabled); + } catch (Exception ex) { + // ignore + } + } + +}
solr/core/src/test/org/apache/solr/util/TestSafeXMLParsing.java+99 −0 added@@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.lang.invoke.MethodHandles; +import java.nio.charset.StandardCharsets; + +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.lucene.util.LuceneTestCase; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.w3c.dom.Document; + +public class TestSafeXMLParsing extends LuceneTestCase { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + public void testUntrusted() throws Exception { + // TODO: Fix the underlying EmptyEntityResolver to not replace external entities by nothing and instead throw exception: + Document doc = SafeXMLParsing.parseUntrustedXML(log, "<!DOCTYPE test [\n" + + "<!ENTITY internalTerm \"foobar\">\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&internalTerm;&externalTerm;</test>"); + assertEquals("foobar", doc.getDocumentElement().getTextContent()); + } + + InputStream getStringStream(String xml) { + return new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)); + } + + public void testConfig() throws Exception { + final ResourceLoader loader = new ResourceLoader() { + @Override + public InputStream openResource(String resource) throws IOException { + switch (resource) { + case "source1.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source2.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"./include1.xml\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source3.xml": + return getStringStream("<foo xmlns:xi=\"http://www.w3.org/2001/XInclude\">\n" + + " <xi:include href=\"./include2.xml\"/>\n" + + "</foo>"); + case "include1.xml": + return getStringStream("Make XML Great Again!™"); + case "include2.xml": + return getStringStream("<bar>Make XML Great Again!™</bar>"); + } + throw new IOException("Resource not found: " + resource); + } + + @Override + public <T> Class<? extends T> findClass(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + @Override + public <T> T newInstance(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + }; + + IOException ioe = expectThrows(IOException.class, () -> { + SafeXMLParsing.parseConfigXML(log, loader, "source1.xml"); + }); + assertTrue(ioe.getMessage().contains("Cannot resolve absolute systemIDs")); + + Document doc = SafeXMLParsing.parseConfigXML(log, loader, "source2.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent()); + + doc = SafeXMLParsing.parseConfigXML(log, loader, "source3.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent().trim()); + } + +}
e5407c5a9710SOLR-12450: Don't allow referal to external resources in various config files
6 files changed · +313 −128
solr/CHANGES.txt+3 −0 modified@@ -358,6 +358,9 @@ Bug Fixes but 'bin/solr create' tells users to use the default action 'set-property', which fails because the property is not editable. (Steve Rowe) +* SOLR-12450: Don't allow referal to external resources in various config files. + (Yuyang Xiao, Uwe Schindler) + Optimizations ----------------------
solr/contrib/extraction/src/java/org/apache/solr/handler/extraction/ParseContextConfig.java+7 −5 modified@@ -16,27 +16,31 @@ */ package org.apache.solr.handler.extraction; -import javax.xml.parsers.DocumentBuilderFactory; import java.beans.BeanInfo; import java.beans.Introspector; import java.beans.PropertyDescriptor; import java.beans.PropertyEditor; import java.beans.PropertyEditorManager; -import java.io.InputStream; +import java.lang.invoke.MethodHandles; import java.lang.reflect.Method; import java.util.HashMap; import java.util.Locale; import java.util.Map; import org.apache.solr.core.SolrResourceLoader; +import org.apache.solr.util.SafeXMLParsing; import org.apache.tika.parser.ParseContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.w3c.dom.Document; import org.w3c.dom.Element; import org.w3c.dom.NamedNodeMap; import org.w3c.dom.Node; import org.w3c.dom.NodeList; public class ParseContextConfig { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + private final Map<Class<?>, Object> entries = new HashMap<>(); /** Creates an empty Config without any settings (used as placeholder). */ @@ -54,9 +58,7 @@ public ParseContextConfig(SolrResourceLoader resourceLoader, String parseContext } private static Document loadConfigFile(SolrResourceLoader resourceLoader, String parseContextConfigLoc) throws Exception { - try (InputStream in = resourceLoader.openResource(parseContextConfigLoc)) { - return DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(in, parseContextConfigLoc); - } + return SafeXMLParsing.parseConfigXML(log, resourceLoader, parseContextConfigLoc); } private void extract(Element element, SolrResourceLoader loader) throws Exception {
solr/core/src/java/org/apache/solr/schema/AbstractEnumField.java+42 −59 modified@@ -18,14 +18,12 @@ package org.apache.solr.schema; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.Collections; import java.util.HashMap; import java.util.Locale; import java.util.Map; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; + import javax.xml.xpath.XPath; import javax.xml.xpath.XPathConstants; import javax.xml.xpath.XPathExpressionException; @@ -38,8 +36,10 @@ import org.apache.lucene.util.BytesRefBuilder; import org.apache.solr.common.EnumFieldValue; import org.apache.solr.common.SolrException; +import org.apache.solr.core.SolrResourceLoader; import org.apache.solr.response.TextResponseWriter; import org.apache.solr.search.QParser; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -51,7 +51,6 @@ * Abstract Field type for support of string values with custom sort order. */ public abstract class AbstractEnumField extends PrimitiveFieldType { - private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); protected EnumMapping enumMapping; @Override @@ -69,6 +68,8 @@ public EnumMapping getEnumMapping() { * @lucene.internal */ public static final class EnumMapping { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + public static final String PARAM_ENUMS_CONFIG = "enumsConfig"; public static final String PARAM_ENUM_NAME = "enumName"; public static final Integer DEFAULT_VALUE = -1; @@ -105,67 +106,49 @@ public EnumMapping(IndexSchema schema, FieldType fieldType, Map<String, String> ftName + ": No enum name was configured."); } - InputStream is = null; - + final SolrResourceLoader loader = schema.getResourceLoader(); try { - is = schema.getResourceLoader().openResource(enumsConfigFile); - final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - final Document doc = dbf.newDocumentBuilder().parse(is); - final XPathFactory xpathFactory = XPathFactory.newInstance(); - final XPath xpath = xpathFactory.newXPath(); - final String xpathStr = String.format(Locale.ROOT, "/enumsConfig/enum[@name='%s']", enumName); - final NodeList nodes = (NodeList) xpath.evaluate(xpathStr, doc, XPathConstants.NODESET); - final int nodesLength = nodes.getLength(); - if (nodesLength == 0) { - String exceptionMessage = String.format - (Locale.ENGLISH, "%s: No enum configuration found for enum '%s' in %s.", + log.debug("Reloading enums config file from {}", enumsConfigFile); + Document doc = SafeXMLParsing.parseConfigXML(log, loader, enumsConfigFile); + final XPathFactory xpathFactory = XPathFactory.newInstance(); + final XPath xpath = xpathFactory.newXPath(); + final String xpathStr = String.format(Locale.ROOT, "/enumsConfig/enum[@name='%s']", enumName); + final NodeList nodes = (NodeList) xpath.evaluate(xpathStr, doc, XPathConstants.NODESET); + final int nodesLength = nodes.getLength(); + if (nodesLength == 0) { + String exceptionMessage = String.format + (Locale.ENGLISH, "%s: No enum configuration found for enum '%s' in %s.", + ftName, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.NOT_FOUND, exceptionMessage); + } + if (nodesLength > 1) { + if (log.isWarnEnabled()) + log.warn("{}: More than one enum configuration found for enum '{}' in {}. The last one was taken.", + ftName, enumName, enumsConfigFile); + } + final Node enumNode = nodes.item(nodesLength - 1); + final NodeList valueNodes = (NodeList) xpath.evaluate("value", enumNode, XPathConstants.NODESET); + for (int i = 0; i < valueNodes.getLength(); i++) { + final Node valueNode = valueNodes.item(i); + final String valueStr = valueNode.getTextContent(); + if ((valueStr == null) || (valueStr.length() == 0)) { + final String exceptionMessage = String.format + (Locale.ENGLISH, "%s: A value was defined with an no value in enum '%s' in %s.", ftName, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.NOT_FOUND, exceptionMessage); - } - if (nodesLength > 1) { - if (log.isWarnEnabled()) - log.warn("{}: More than one enum configuration found for enum '{}' in {}. The last one was taken.", - ftName, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); } - final Node enumNode = nodes.item(nodesLength - 1); - final NodeList valueNodes = (NodeList) xpath.evaluate("value", enumNode, XPathConstants.NODESET); - for (int i = 0; i < valueNodes.getLength(); i++) { - final Node valueNode = valueNodes.item(i); - final String valueStr = valueNode.getTextContent(); - if ((valueStr == null) || (valueStr.length() == 0)) { - final String exceptionMessage = String.format - (Locale.ENGLISH, "%s: A value was defined with an no value in enum '%s' in %s.", - ftName, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); - } - if (enumStringToIntMap.containsKey(valueStr)) { - final String exceptionMessage = String.format - (Locale.ENGLISH, "%s: A duplicated definition was found for value '%s' in enum '%s' in %s.", - ftName, valueStr, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); - } - enumIntToStringMap.put(i, valueStr); - enumStringToIntMap.put(valueStr, i); + if (enumStringToIntMap.containsKey(valueStr)) { + final String exceptionMessage = String.format + (Locale.ENGLISH, "%s: A duplicated definition was found for value '%s' in enum '%s' in %s.", + ftName, valueStr, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); } + enumIntToStringMap.put(i, valueStr); + enumStringToIntMap.put(valueStr, i); } - catch (ParserConfigurationException | XPathExpressionException | SAXException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, - ftName + ": Error parsing enums config.", e); - } - } - catch (IOException e) { + } catch (IOException | SAXException | XPathExpressionException e) { throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, - ftName + ": Error while opening enums config.", e); - } finally { - try { - if (is != null) { - is.close(); - } - } - catch (IOException e) { - e.printStackTrace(); - } + ftName + ": Error while parsing enums config.", e); } if ((enumStringToIntMap.size() == 0) || (enumIntToStringMap.size() == 0)) {
solr/core/src/java/org/apache/solr/schema/FileExchangeRateProvider.java+42 −64 modified@@ -17,22 +17,21 @@ package org.apache.solr.schema; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; -import javax.xml.xpath.XPath; -import javax.xml.xpath.XPathConstants; -import javax.xml.xpath.XPathExpressionException; -import javax.xml.xpath.XPathFactory; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Set; +import javax.xml.xpath.XPath; +import javax.xml.xpath.XPathConstants; +import javax.xml.xpath.XPathExpressionException; +import javax.xml.xpath.XPathFactory; + import org.apache.lucene.analysis.util.ResourceLoader; import org.apache.solr.common.SolrException; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -46,6 +45,7 @@ */ class FileExchangeRateProvider implements ExchangeRateProvider { private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + protected static final String PARAM_CURRENCY_CONFIG = "currencyConfig"; // Exchange rate map, maps Currency Code -> Currency Code -> Rate @@ -159,71 +159,49 @@ public Set<String> listAvailableCurrencies() { @Override public boolean reload() throws SolrException { - InputStream is = null; Map<String, Map<String, Double>> tmpRates = new HashMap<>(); + log.debug("Reloading exchange rates from file {}", currencyConfigFile); + try { - log.debug("Reloading exchange rates from file "+this.currencyConfigFile); - - is = loader.openResource(currencyConfigFile); - javax.xml.parsers.DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - dbf.setXIncludeAware(true); - dbf.setNamespaceAware(true); - } catch (UnsupportedOperationException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); - } + Document doc = SafeXMLParsing.parseConfigXML(log, loader, currencyConfigFile); + XPathFactory xpathFactory = XPathFactory.newInstance(); + XPath xpath = xpathFactory.newXPath(); + + // Parse exchange rates. + NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); - try { - Document doc = dbf.newDocumentBuilder().parse(is); - XPathFactory xpathFactory = XPathFactory.newInstance(); - XPath xpath = xpathFactory.newXPath(); + for (int i = 0; i < nodes.getLength(); i++) { + Node rateNode = nodes.item(i); + NamedNodeMap attributes = rateNode.getAttributes(); + Node from = attributes.getNamedItem("from"); + Node to = attributes.getNamedItem("to"); + Node rate = attributes.getNamedItem("rate"); + + if (from == null || to == null || rate == null) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); + } - // Parse exchange rates. - NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); + String fromCurrency = from.getNodeValue(); + String toCurrency = to.getNodeValue(); + Double exchangeRate; - for (int i = 0; i < nodes.getLength(); i++) { - Node rateNode = nodes.item(i); - NamedNodeMap attributes = rateNode.getAttributes(); - Node from = attributes.getNamedItem("from"); - Node to = attributes.getNamedItem("to"); - Node rate = attributes.getNamedItem("rate"); - - if (from == null || to == null || rate == null) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); - } - - String fromCurrency = from.getNodeValue(); - String toCurrency = to.getNodeValue(); - Double exchangeRate; - - if (null == CurrencyFieldType.getCurrency(fromCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); - } - if (null == CurrencyFieldType.getCurrency(toCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); - } - - try { - exchangeRate = Double.parseDouble(rate.getNodeValue()); - } catch (NumberFormatException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); - } - - addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); + if (null == CurrencyFieldType.getCurrency(fromCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); } - } catch (SAXException | XPathExpressionException | ParserConfigurationException | IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error parsing currency config.", e); - } - } catch (IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while opening Currency configuration file "+currencyConfigFile, e); - } finally { - try { - if (is != null) { - is.close(); + if (null == CurrencyFieldType.getCurrency(toCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); + } + + try { + exchangeRate = Double.parseDouble(rate.getNodeValue()); + } catch (NumberFormatException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); } - } catch (IOException e) { - e.printStackTrace(); + + addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); } + } catch (SAXException | IOException | XPathExpressionException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while parsing currency configuration file "+currencyConfigFile, e); } // Atomically swap in the new rates map, if it loaded successfully this.rates = tmpRates;
solr/core/src/java/org/apache/solr/util/SafeXMLParsing.java+120 −0 added@@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.FilterReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.io.StringReader; + +import javax.xml.XMLConstants; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.parsers.ParserConfigurationException; + +import org.apache.commons.io.input.CloseShieldInputStream; +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.solr.common.EmptyEntityResolver; +import org.apache.solr.common.SolrException; +import org.apache.solr.common.util.SuppressForbidden; +import org.apache.solr.common.util.XMLErrorLogger; +import org.slf4j.Logger; +import org.w3c.dom.Document; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; + +/** + * Some utility methods for parsing XML in a safe way. This class can be used to parse XML + * coming from network (completely untrusted) or it can load a config file from a + * {@link ResourceLoader}. In this case it allows external entities and xincludes, but only + * referring to files reachable by the loader. + */ +@SuppressForbidden(reason = "This class uses XML APIs directly that should not be used anywhere else in Solr code") +public final class SafeXMLParsing { + + public static final String SYSTEMID_UNTRUSTED = "untrusted://stream"; + + private SafeXMLParsing() {} + + /** Parses a config file from ResourceLoader. Xinclude and external entities are enabled, but cannot escape the resource loader. */ + public static Document parseConfigXML(Logger log, ResourceLoader loader, String file) throws SAXException, IOException { + try (InputStream in = loader.openResource(file)) { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + try { + dbf.setXIncludeAware(true); + } catch (UnsupportedOperationException e) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); + } + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(new SystemIdResolver(loader)); + db.setErrorHandler(new XMLErrorLogger(log)); + return db.parse(in, SystemIdResolver.createSystemIdFromResourceName(file)); + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given InputStream is not closed. */ + public static Document parseUntrustedXML(Logger log, InputStream in) throws SAXException, IOException { + return getUntrustedDocumentBuilder(log).parse(new CloseShieldInputStream(in), SYSTEMID_UNTRUSTED); + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given Reader is not closed. */ + public static Document parseUntrustedXML(Logger log, Reader reader) throws SAXException, IOException { + final InputSource is = new InputSource(new FilterReader(reader) { + @Override public void close() {} + }); + is.setSystemId(SYSTEMID_UNTRUSTED); + return getUntrustedDocumentBuilder(log).parse(is); + } + + public static Document parseUntrustedXML(Logger log, String xml) throws SAXException, IOException { + return parseUntrustedXML(log, new StringReader(xml)); + } + + private static DocumentBuilder getUntrustedDocumentBuilder(Logger log) { + try { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE); + db.setErrorHandler(new XMLErrorLogger(log)); + return db; + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + private static void trySetDOMFeature(DocumentBuilderFactory factory, String feature, boolean enabled) { + try { + factory.setFeature(feature, enabled); + } catch (Exception ex) { + // ignore + } + } + +}
solr/core/src/test/org/apache/solr/util/TestSafeXMLParsing.java+99 −0 added@@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.lang.invoke.MethodHandles; +import java.nio.charset.StandardCharsets; + +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.lucene.util.LuceneTestCase; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.w3c.dom.Document; + +public class TestSafeXMLParsing extends LuceneTestCase { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + public void testUntrusted() throws Exception { + // TODO: Fix the underlying EmptyEntityResolver to not replace external entities by nothing and instead throw exception: + Document doc = SafeXMLParsing.parseUntrustedXML(log, "<!DOCTYPE test [\n" + + "<!ENTITY internalTerm \"foobar\">\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&internalTerm;&externalTerm;</test>"); + assertEquals("foobar", doc.getDocumentElement().getTextContent()); + } + + InputStream getStringStream(String xml) { + return new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)); + } + + public void testConfig() throws Exception { + final ResourceLoader loader = new ResourceLoader() { + @Override + public InputStream openResource(String resource) throws IOException { + switch (resource) { + case "source1.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source2.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"./include1.xml\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source3.xml": + return getStringStream("<foo xmlns:xi=\"http://www.w3.org/2001/XInclude\">\n" + + " <xi:include href=\"./include2.xml\"/>\n" + + "</foo>"); + case "include1.xml": + return getStringStream("Make XML Great Again!™"); + case "include2.xml": + return getStringStream("<bar>Make XML Great Again!™</bar>"); + } + throw new IOException("Resource not found: " + resource); + } + + @Override + public <T> Class<? extends T> findClass(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + @Override + public <T> T newInstance(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + }; + + IOException ioe = expectThrows(IOException.class, () -> { + SafeXMLParsing.parseConfigXML(log, loader, "source1.xml"); + }); + assertTrue(ioe.getMessage().contains("Cannot resolve absolute systemIDs")); + + Document doc = SafeXMLParsing.parseConfigXML(log, loader, "source2.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent()); + + doc = SafeXMLParsing.parseConfigXML(log, loader, "source3.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent().trim()); + } + +}
3aa6086ed99fSOLR-12450: Don't allow referal to external resources in various config files
6 files changed · +313 −128
solr/CHANGES.txt+3 −0 modified@@ -297,6 +297,9 @@ Bug Fixes * SOLR-12416: When creating a time routed alias, the router.autoDeleteAge option wasn't considered. (Joachim Sauer via David Smiley) +* SOLR-12450: Don't allow referal to external resources in various config files. + (Yuyang Xiao, Uwe Schindler) + Optimizations ----------------------
solr/contrib/extraction/src/java/org/apache/solr/handler/extraction/ParseContextConfig.java+7 −5 modified@@ -16,27 +16,31 @@ */ package org.apache.solr.handler.extraction; -import javax.xml.parsers.DocumentBuilderFactory; import java.beans.BeanInfo; import java.beans.Introspector; import java.beans.PropertyDescriptor; import java.beans.PropertyEditor; import java.beans.PropertyEditorManager; -import java.io.InputStream; +import java.lang.invoke.MethodHandles; import java.lang.reflect.Method; import java.util.HashMap; import java.util.Locale; import java.util.Map; import org.apache.solr.core.SolrResourceLoader; +import org.apache.solr.util.SafeXMLParsing; import org.apache.tika.parser.ParseContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.w3c.dom.Document; import org.w3c.dom.Element; import org.w3c.dom.NamedNodeMap; import org.w3c.dom.Node; import org.w3c.dom.NodeList; public class ParseContextConfig { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + private final Map<Class<?>, Object> entries = new HashMap<>(); /** Creates an empty Config without any settings (used as placeholder). */ @@ -54,9 +58,7 @@ public ParseContextConfig(SolrResourceLoader resourceLoader, String parseContext } private static Document loadConfigFile(SolrResourceLoader resourceLoader, String parseContextConfigLoc) throws Exception { - try (InputStream in = resourceLoader.openResource(parseContextConfigLoc)) { - return DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(in, parseContextConfigLoc); - } + return SafeXMLParsing.parseConfigXML(log, resourceLoader, parseContextConfigLoc); } private void extract(Element element, SolrResourceLoader loader) throws Exception {
solr/core/src/java/org/apache/solr/schema/AbstractEnumField.java+42 −59 modified@@ -18,14 +18,12 @@ package org.apache.solr.schema; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.Collections; import java.util.HashMap; import java.util.Locale; import java.util.Map; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; + import javax.xml.xpath.XPath; import javax.xml.xpath.XPathConstants; import javax.xml.xpath.XPathExpressionException; @@ -38,8 +36,10 @@ import org.apache.lucene.util.BytesRefBuilder; import org.apache.solr.common.EnumFieldValue; import org.apache.solr.common.SolrException; +import org.apache.solr.core.SolrResourceLoader; import org.apache.solr.response.TextResponseWriter; import org.apache.solr.search.QParser; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -51,7 +51,6 @@ * Abstract Field type for support of string values with custom sort order. */ public abstract class AbstractEnumField extends PrimitiveFieldType { - private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); protected EnumMapping enumMapping; @Override @@ -69,6 +68,8 @@ public EnumMapping getEnumMapping() { * @lucene.internal */ public static final class EnumMapping { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + public static final String PARAM_ENUMS_CONFIG = "enumsConfig"; public static final String PARAM_ENUM_NAME = "enumName"; public static final Integer DEFAULT_VALUE = -1; @@ -105,67 +106,49 @@ public EnumMapping(IndexSchema schema, FieldType fieldType, Map<String, String> ftName + ": No enum name was configured."); } - InputStream is = null; - + final SolrResourceLoader loader = schema.getResourceLoader(); try { - is = schema.getResourceLoader().openResource(enumsConfigFile); - final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - final Document doc = dbf.newDocumentBuilder().parse(is); - final XPathFactory xpathFactory = XPathFactory.newInstance(); - final XPath xpath = xpathFactory.newXPath(); - final String xpathStr = String.format(Locale.ROOT, "/enumsConfig/enum[@name='%s']", enumName); - final NodeList nodes = (NodeList) xpath.evaluate(xpathStr, doc, XPathConstants.NODESET); - final int nodesLength = nodes.getLength(); - if (nodesLength == 0) { - String exceptionMessage = String.format - (Locale.ENGLISH, "%s: No enum configuration found for enum '%s' in %s.", + log.debug("Reloading enums config file from {}", enumsConfigFile); + Document doc = SafeXMLParsing.parseConfigXML(log, loader, enumsConfigFile); + final XPathFactory xpathFactory = XPathFactory.newInstance(); + final XPath xpath = xpathFactory.newXPath(); + final String xpathStr = String.format(Locale.ROOT, "/enumsConfig/enum[@name='%s']", enumName); + final NodeList nodes = (NodeList) xpath.evaluate(xpathStr, doc, XPathConstants.NODESET); + final int nodesLength = nodes.getLength(); + if (nodesLength == 0) { + String exceptionMessage = String.format + (Locale.ENGLISH, "%s: No enum configuration found for enum '%s' in %s.", + ftName, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.NOT_FOUND, exceptionMessage); + } + if (nodesLength > 1) { + if (log.isWarnEnabled()) + log.warn("{}: More than one enum configuration found for enum '{}' in {}. The last one was taken.", + ftName, enumName, enumsConfigFile); + } + final Node enumNode = nodes.item(nodesLength - 1); + final NodeList valueNodes = (NodeList) xpath.evaluate("value", enumNode, XPathConstants.NODESET); + for (int i = 0; i < valueNodes.getLength(); i++) { + final Node valueNode = valueNodes.item(i); + final String valueStr = valueNode.getTextContent(); + if ((valueStr == null) || (valueStr.length() == 0)) { + final String exceptionMessage = String.format + (Locale.ENGLISH, "%s: A value was defined with an no value in enum '%s' in %s.", ftName, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.NOT_FOUND, exceptionMessage); - } - if (nodesLength > 1) { - if (log.isWarnEnabled()) - log.warn("{}: More than one enum configuration found for enum '{}' in {}. The last one was taken.", - ftName, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); } - final Node enumNode = nodes.item(nodesLength - 1); - final NodeList valueNodes = (NodeList) xpath.evaluate("value", enumNode, XPathConstants.NODESET); - for (int i = 0; i < valueNodes.getLength(); i++) { - final Node valueNode = valueNodes.item(i); - final String valueStr = valueNode.getTextContent(); - if ((valueStr == null) || (valueStr.length() == 0)) { - final String exceptionMessage = String.format - (Locale.ENGLISH, "%s: A value was defined with an no value in enum '%s' in %s.", - ftName, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); - } - if (enumStringToIntMap.containsKey(valueStr)) { - final String exceptionMessage = String.format - (Locale.ENGLISH, "%s: A duplicated definition was found for value '%s' in enum '%s' in %s.", - ftName, valueStr, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); - } - enumIntToStringMap.put(i, valueStr); - enumStringToIntMap.put(valueStr, i); + if (enumStringToIntMap.containsKey(valueStr)) { + final String exceptionMessage = String.format + (Locale.ENGLISH, "%s: A duplicated definition was found for value '%s' in enum '%s' in %s.", + ftName, valueStr, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); } + enumIntToStringMap.put(i, valueStr); + enumStringToIntMap.put(valueStr, i); } - catch (ParserConfigurationException | XPathExpressionException | SAXException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, - ftName + ": Error parsing enums config.", e); - } - } - catch (IOException e) { + } catch (IOException | SAXException | XPathExpressionException e) { throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, - ftName + ": Error while opening enums config.", e); - } finally { - try { - if (is != null) { - is.close(); - } - } - catch (IOException e) { - e.printStackTrace(); - } + ftName + ": Error while parsing enums config.", e); } if ((enumStringToIntMap.size() == 0) || (enumIntToStringMap.size() == 0)) {
solr/core/src/java/org/apache/solr/schema/FileExchangeRateProvider.java+42 −64 modified@@ -17,22 +17,21 @@ package org.apache.solr.schema; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; -import javax.xml.xpath.XPath; -import javax.xml.xpath.XPathConstants; -import javax.xml.xpath.XPathExpressionException; -import javax.xml.xpath.XPathFactory; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Set; +import javax.xml.xpath.XPath; +import javax.xml.xpath.XPathConstants; +import javax.xml.xpath.XPathExpressionException; +import javax.xml.xpath.XPathFactory; + import org.apache.lucene.analysis.util.ResourceLoader; import org.apache.solr.common.SolrException; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -46,6 +45,7 @@ */ class FileExchangeRateProvider implements ExchangeRateProvider { private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + protected static final String PARAM_CURRENCY_CONFIG = "currencyConfig"; // Exchange rate map, maps Currency Code -> Currency Code -> Rate @@ -159,71 +159,49 @@ public Set<String> listAvailableCurrencies() { @Override public boolean reload() throws SolrException { - InputStream is = null; Map<String, Map<String, Double>> tmpRates = new HashMap<>(); + log.debug("Reloading exchange rates from file {}", currencyConfigFile); + try { - log.debug("Reloading exchange rates from file "+this.currencyConfigFile); - - is = loader.openResource(currencyConfigFile); - javax.xml.parsers.DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - dbf.setXIncludeAware(true); - dbf.setNamespaceAware(true); - } catch (UnsupportedOperationException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); - } + Document doc = SafeXMLParsing.parseConfigXML(log, loader, currencyConfigFile); + XPathFactory xpathFactory = XPathFactory.newInstance(); + XPath xpath = xpathFactory.newXPath(); + + // Parse exchange rates. + NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); - try { - Document doc = dbf.newDocumentBuilder().parse(is); - XPathFactory xpathFactory = XPathFactory.newInstance(); - XPath xpath = xpathFactory.newXPath(); + for (int i = 0; i < nodes.getLength(); i++) { + Node rateNode = nodes.item(i); + NamedNodeMap attributes = rateNode.getAttributes(); + Node from = attributes.getNamedItem("from"); + Node to = attributes.getNamedItem("to"); + Node rate = attributes.getNamedItem("rate"); + + if (from == null || to == null || rate == null) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); + } - // Parse exchange rates. - NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); + String fromCurrency = from.getNodeValue(); + String toCurrency = to.getNodeValue(); + Double exchangeRate; - for (int i = 0; i < nodes.getLength(); i++) { - Node rateNode = nodes.item(i); - NamedNodeMap attributes = rateNode.getAttributes(); - Node from = attributes.getNamedItem("from"); - Node to = attributes.getNamedItem("to"); - Node rate = attributes.getNamedItem("rate"); - - if (from == null || to == null || rate == null) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); - } - - String fromCurrency = from.getNodeValue(); - String toCurrency = to.getNodeValue(); - Double exchangeRate; - - if (null == CurrencyFieldType.getCurrency(fromCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); - } - if (null == CurrencyFieldType.getCurrency(toCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); - } - - try { - exchangeRate = Double.parseDouble(rate.getNodeValue()); - } catch (NumberFormatException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); - } - - addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); + if (null == CurrencyFieldType.getCurrency(fromCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); } - } catch (SAXException | XPathExpressionException | ParserConfigurationException | IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error parsing currency config.", e); - } - } catch (IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while opening Currency configuration file "+currencyConfigFile, e); - } finally { - try { - if (is != null) { - is.close(); + if (null == CurrencyFieldType.getCurrency(toCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); + } + + try { + exchangeRate = Double.parseDouble(rate.getNodeValue()); + } catch (NumberFormatException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); } - } catch (IOException e) { - e.printStackTrace(); + + addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); } + } catch (SAXException | IOException | XPathExpressionException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while parsing currency configuration file "+currencyConfigFile, e); } // Atomically swap in the new rates map, if it loaded successfully this.rates = tmpRates;
solr/core/src/java/org/apache/solr/util/SafeXMLParsing.java+120 −0 added@@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.FilterReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.io.StringReader; + +import javax.xml.XMLConstants; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.parsers.ParserConfigurationException; + +import org.apache.commons.io.input.CloseShieldInputStream; +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.solr.common.EmptyEntityResolver; +import org.apache.solr.common.SolrException; +import org.apache.solr.common.util.SuppressForbidden; +import org.apache.solr.common.util.XMLErrorLogger; +import org.slf4j.Logger; +import org.w3c.dom.Document; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; + +/** + * Some utility methods for parsing XML in a safe way. This class can be used to parse XML + * coming from network (completely untrusted) or it can load a config file from a + * {@link ResourceLoader}. In this case it allows external entities and xincludes, but only + * referring to files reachable by the loader. + */ +@SuppressForbidden(reason = "This class uses XML APIs directly that should not be used anywhere else in Solr code") +public final class SafeXMLParsing { + + public static final String SYSTEMID_UNTRUSTED = "untrusted://stream"; + + private SafeXMLParsing() {} + + /** Parses a config file from ResourceLoader. Xinclude and external entities are enabled, but cannot escape the resource loader. */ + public static Document parseConfigXML(Logger log, ResourceLoader loader, String file) throws SAXException, IOException { + try (InputStream in = loader.openResource(file)) { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + try { + dbf.setXIncludeAware(true); + } catch (UnsupportedOperationException e) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); + } + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(new SystemIdResolver(loader)); + db.setErrorHandler(new XMLErrorLogger(log)); + return db.parse(in, SystemIdResolver.createSystemIdFromResourceName(file)); + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given InputStream is not closed. */ + public static Document parseUntrustedXML(Logger log, InputStream in) throws SAXException, IOException { + return getUntrustedDocumentBuilder(log).parse(new CloseShieldInputStream(in), SYSTEMID_UNTRUSTED); + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given Reader is not closed. */ + public static Document parseUntrustedXML(Logger log, Reader reader) throws SAXException, IOException { + final InputSource is = new InputSource(new FilterReader(reader) { + @Override public void close() {} + }); + is.setSystemId(SYSTEMID_UNTRUSTED); + return getUntrustedDocumentBuilder(log).parse(is); + } + + public static Document parseUntrustedXML(Logger log, String xml) throws SAXException, IOException { + return parseUntrustedXML(log, new StringReader(xml)); + } + + private static DocumentBuilder getUntrustedDocumentBuilder(Logger log) { + try { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE); + db.setErrorHandler(new XMLErrorLogger(log)); + return db; + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + private static void trySetDOMFeature(DocumentBuilderFactory factory, String feature, boolean enabled) { + try { + factory.setFeature(feature, enabled); + } catch (Exception ex) { + // ignore + } + } + +}
solr/core/src/test/org/apache/solr/util/TestSafeXMLParsing.java+99 −0 added@@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.lang.invoke.MethodHandles; +import java.nio.charset.StandardCharsets; + +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.lucene.util.LuceneTestCase; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.w3c.dom.Document; + +public class TestSafeXMLParsing extends LuceneTestCase { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + public void testUntrusted() throws Exception { + // TODO: Fix the underlying EmptyEntityResolver to not replace external entities by nothing and instead throw exception: + Document doc = SafeXMLParsing.parseUntrustedXML(log, "<!DOCTYPE test [\n" + + "<!ENTITY internalTerm \"foobar\">\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&internalTerm;&externalTerm;</test>"); + assertEquals("foobar", doc.getDocumentElement().getTextContent()); + } + + InputStream getStringStream(String xml) { + return new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)); + } + + public void testConfig() throws Exception { + final ResourceLoader loader = new ResourceLoader() { + @Override + public InputStream openResource(String resource) throws IOException { + switch (resource) { + case "source1.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source2.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"./include1.xml\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source3.xml": + return getStringStream("<foo xmlns:xi=\"http://www.w3.org/2001/XInclude\">\n" + + " <xi:include href=\"./include2.xml\"/>\n" + + "</foo>"); + case "include1.xml": + return getStringStream("Make XML Great Again!™"); + case "include2.xml": + return getStringStream("<bar>Make XML Great Again!™</bar>"); + } + throw new IOException("Resource not found: " + resource); + } + + @Override + public <T> Class<? extends T> findClass(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + @Override + public <T> T newInstance(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + }; + + IOException ioe = expectThrows(IOException.class, () -> { + SafeXMLParsing.parseConfigXML(log, loader, "source1.xml"); + }); + assertTrue(ioe.getMessage().contains("Cannot resolve absolute systemIDs")); + + Document doc = SafeXMLParsing.parseConfigXML(log, loader, "source2.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent()); + + doc = SafeXMLParsing.parseConfigXML(log, loader, "source3.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent().trim()); + } + +}
d1baf6ba5935SOLR-12450: Don't allow referal to external resources in various config files
4 files changed · +245 −5
solr/CHANGES.txt+19 −0 modified@@ -16,6 +16,25 @@ In this release, there is an example Solr server including a bundled servlet container in the directory named "example". See the Quick Start guide at http://lucene.apache.org/solr/quickstart.html +================== 6.6.5 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.14.v20161028 + +Bug Fixes +---------------------- + + * SOLR-12450: Don't allow referal to external resources in various config files. + (Yuyang Xiao, Uwe Schindler) + ================== 6.6.4 ================== Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
solr/contrib/extraction/src/java/org/apache/solr/handler/extraction/ParseContextConfig.java+7 −5 modified@@ -16,27 +16,31 @@ */ package org.apache.solr.handler.extraction; -import javax.xml.parsers.DocumentBuilderFactory; import java.beans.BeanInfo; import java.beans.Introspector; import java.beans.PropertyDescriptor; import java.beans.PropertyEditor; import java.beans.PropertyEditorManager; -import java.io.InputStream; +import java.lang.invoke.MethodHandles; import java.lang.reflect.Method; import java.util.HashMap; import java.util.Locale; import java.util.Map; import org.apache.solr.core.SolrResourceLoader; +import org.apache.solr.util.SafeXMLParsing; import org.apache.tika.parser.ParseContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.w3c.dom.Document; import org.w3c.dom.Element; import org.w3c.dom.NamedNodeMap; import org.w3c.dom.Node; import org.w3c.dom.NodeList; public class ParseContextConfig { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + private final Map<Class<?>, Object> entries = new HashMap<>(); /** Creates an empty Config without any settings (used as placeholder). */ @@ -54,9 +58,7 @@ public ParseContextConfig(SolrResourceLoader resourceLoader, String parseContext } private static Document loadConfigFile(SolrResourceLoader resourceLoader, String parseContextConfigLoc) throws Exception { - try (InputStream in = resourceLoader.openResource(parseContextConfigLoc)) { - return DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(in, parseContextConfigLoc); - } + return SafeXMLParsing.parseConfigXML(log, resourceLoader, parseContextConfigLoc); } private void extract(Element element, SolrResourceLoader loader) throws Exception {
solr/core/src/java/org/apache/solr/util/SafeXMLParsing.java+120 −0 added@@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.FilterReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.io.StringReader; + +import javax.xml.XMLConstants; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.parsers.ParserConfigurationException; + +import org.apache.commons.io.input.CloseShieldInputStream; +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.solr.common.EmptyEntityResolver; +import org.apache.solr.common.SolrException; +import org.apache.solr.common.util.SuppressForbidden; +import org.apache.solr.common.util.XMLErrorLogger; +import org.slf4j.Logger; +import org.w3c.dom.Document; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; + +/** + * Some utility methods for parsing XML in a safe way. This class can be used to parse XML + * coming from network (completely untrusted) or it can load a config file from a + * {@link ResourceLoader}. In this case it allows external entities and xincludes, but only + * referring to files reachable by the loader. + */ +@SuppressForbidden(reason = "This class uses XML APIs directly that should not be used anywhere else in Solr code") +public final class SafeXMLParsing { + + public static final String SYSTEMID_UNTRUSTED = "untrusted://stream"; + + private SafeXMLParsing() {} + + /** Parses a config file from ResourceLoader. Xinclude and external entities are enabled, but cannot escape the resource loader. */ + public static Document parseConfigXML(Logger log, ResourceLoader loader, String file) throws SAXException, IOException { + try (InputStream in = loader.openResource(file)) { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + try { + dbf.setXIncludeAware(true); + } catch (UnsupportedOperationException e) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); + } + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(new SystemIdResolver(loader)); + db.setErrorHandler(new XMLErrorLogger(log)); + return db.parse(in, SystemIdResolver.createSystemIdFromResourceName(file)); + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given InputStream is not closed. */ + public static Document parseUntrustedXML(Logger log, InputStream in) throws SAXException, IOException { + return getUntrustedDocumentBuilder(log).parse(new CloseShieldInputStream(in), SYSTEMID_UNTRUSTED); + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given Reader is not closed. */ + public static Document parseUntrustedXML(Logger log, Reader reader) throws SAXException, IOException { + final InputSource is = new InputSource(new FilterReader(reader) { + @Override public void close() {} + }); + is.setSystemId(SYSTEMID_UNTRUSTED); + return getUntrustedDocumentBuilder(log).parse(is); + } + + public static Document parseUntrustedXML(Logger log, String xml) throws SAXException, IOException { + return parseUntrustedXML(log, new StringReader(xml)); + } + + private static DocumentBuilder getUntrustedDocumentBuilder(Logger log) { + try { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE); + db.setErrorHandler(new XMLErrorLogger(log)); + return db; + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + private static void trySetDOMFeature(DocumentBuilderFactory factory, String feature, boolean enabled) { + try { + factory.setFeature(feature, enabled); + } catch (Exception ex) { + // ignore + } + } + +}
solr/core/src/test/org/apache/solr/util/TestSafeXMLParsing.java+99 −0 added@@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.lang.invoke.MethodHandles; +import java.nio.charset.StandardCharsets; + +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.lucene.util.LuceneTestCase; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.w3c.dom.Document; + +public class TestSafeXMLParsing extends LuceneTestCase { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + public void testUntrusted() throws Exception { + // TODO: Fix the underlying EmptyEntityResolver to not replace external entities by nothing and instead throw exception: + Document doc = SafeXMLParsing.parseUntrustedXML(log, "<!DOCTYPE test [\n" + + "<!ENTITY internalTerm \"foobar\">\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&internalTerm;&externalTerm;</test>"); + assertEquals("foobar", doc.getDocumentElement().getTextContent()); + } + + InputStream getStringStream(String xml) { + return new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)); + } + + public void testConfig() throws Exception { + final ResourceLoader loader = new ResourceLoader() { + @Override + public InputStream openResource(String resource) throws IOException { + switch (resource) { + case "source1.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source2.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"./include1.xml\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source3.xml": + return getStringStream("<foo xmlns:xi=\"http://www.w3.org/2001/XInclude\">\n" + + " <xi:include href=\"./include2.xml\"/>\n" + + "</foo>"); + case "include1.xml": + return getStringStream("Make XML Great Again!™"); + case "include2.xml": + return getStringStream("<bar>Make XML Great Again!™</bar>"); + } + throw new IOException("Resource not found: " + resource); + } + + @Override + public <T> Class<? extends T> findClass(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + @Override + public <T> T newInstance(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + }; + + IOException ioe = expectThrows(IOException.class, () -> { + SafeXMLParsing.parseConfigXML(log, loader, "source1.xml"); + }); + assertTrue(ioe.getMessage().contains("Cannot resolve absolute systemIDs")); + + Document doc = SafeXMLParsing.parseConfigXML(log, loader, "source2.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent()); + + doc = SafeXMLParsing.parseConfigXML(log, loader, "source3.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent().trim()); + } + +}
1880d4824e6cSOLR-12450: Don't allow referal to external resources in various config files
5 files changed · +287 −69
solr/CHANGES.txt+19 −0 modified@@ -221,6 +221,25 @@ Other Changes * SOLR-11122: Creating a core should write a core.properties file first and clean up on failure (Erick Erickson) +================== 6.6.5 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.14.v20161028 + +Bug Fixes +---------------------- + + * SOLR-12450: Don't allow referal to external resources in various config files. + (Yuyang Xiao, Uwe Schindler) + ================== 6.6.4 ================== Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
solr/contrib/extraction/src/java/org/apache/solr/handler/extraction/ParseContextConfig.java+7 −5 modified@@ -16,27 +16,31 @@ */ package org.apache.solr.handler.extraction; -import javax.xml.parsers.DocumentBuilderFactory; import java.beans.BeanInfo; import java.beans.Introspector; import java.beans.PropertyDescriptor; import java.beans.PropertyEditor; import java.beans.PropertyEditorManager; -import java.io.InputStream; +import java.lang.invoke.MethodHandles; import java.lang.reflect.Method; import java.util.HashMap; import java.util.Locale; import java.util.Map; import org.apache.solr.core.SolrResourceLoader; +import org.apache.solr.util.SafeXMLParsing; import org.apache.tika.parser.ParseContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.w3c.dom.Document; import org.w3c.dom.Element; import org.w3c.dom.NamedNodeMap; import org.w3c.dom.Node; import org.w3c.dom.NodeList; public class ParseContextConfig { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + private final Map<Class<?>, Object> entries = new HashMap<>(); /** Creates an empty Config without any settings (used as placeholder). */ @@ -54,9 +58,7 @@ public ParseContextConfig(SolrResourceLoader resourceLoader, String parseContext } private static Document loadConfigFile(SolrResourceLoader resourceLoader, String parseContextConfigLoc) throws Exception { - try (InputStream in = resourceLoader.openResource(parseContextConfigLoc)) { - return DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(in, parseContextConfigLoc); - } + return SafeXMLParsing.parseConfigXML(log, resourceLoader, parseContextConfigLoc); } private void extract(Element element, SolrResourceLoader loader) throws Exception {
solr/core/src/java/org/apache/solr/schema/FileExchangeRateProvider.java+42 −64 modified@@ -17,22 +17,21 @@ package org.apache.solr.schema; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; -import javax.xml.xpath.XPath; -import javax.xml.xpath.XPathConstants; -import javax.xml.xpath.XPathExpressionException; -import javax.xml.xpath.XPathFactory; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Set; +import javax.xml.xpath.XPath; +import javax.xml.xpath.XPathConstants; +import javax.xml.xpath.XPathExpressionException; +import javax.xml.xpath.XPathFactory; + import org.apache.lucene.analysis.util.ResourceLoader; import org.apache.solr.common.SolrException; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -46,6 +45,7 @@ */ class FileExchangeRateProvider implements ExchangeRateProvider { private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + protected static final String PARAM_CURRENCY_CONFIG = "currencyConfig"; // Exchange rate map, maps Currency Code -> Currency Code -> Rate @@ -159,71 +159,49 @@ public Set<String> listAvailableCurrencies() { @Override public boolean reload() throws SolrException { - InputStream is = null; Map<String, Map<String, Double>> tmpRates = new HashMap<>(); + log.debug("Reloading exchange rates from file {}", currencyConfigFile); + try { - log.debug("Reloading exchange rates from file "+this.currencyConfigFile); - - is = loader.openResource(currencyConfigFile); - javax.xml.parsers.DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - dbf.setXIncludeAware(true); - dbf.setNamespaceAware(true); - } catch (UnsupportedOperationException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); - } + Document doc = SafeXMLParsing.parseConfigXML(log, loader, currencyConfigFile); + XPathFactory xpathFactory = XPathFactory.newInstance(); + XPath xpath = xpathFactory.newXPath(); + + // Parse exchange rates. + NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); - try { - Document doc = dbf.newDocumentBuilder().parse(is); - XPathFactory xpathFactory = XPathFactory.newInstance(); - XPath xpath = xpathFactory.newXPath(); + for (int i = 0; i < nodes.getLength(); i++) { + Node rateNode = nodes.item(i); + NamedNodeMap attributes = rateNode.getAttributes(); + Node from = attributes.getNamedItem("from"); + Node to = attributes.getNamedItem("to"); + Node rate = attributes.getNamedItem("rate"); + + if (from == null || to == null || rate == null) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); + } - // Parse exchange rates. - NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); + String fromCurrency = from.getNodeValue(); + String toCurrency = to.getNodeValue(); + Double exchangeRate; - for (int i = 0; i < nodes.getLength(); i++) { - Node rateNode = nodes.item(i); - NamedNodeMap attributes = rateNode.getAttributes(); - Node from = attributes.getNamedItem("from"); - Node to = attributes.getNamedItem("to"); - Node rate = attributes.getNamedItem("rate"); - - if (from == null || to == null || rate == null) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); - } - - String fromCurrency = from.getNodeValue(); - String toCurrency = to.getNodeValue(); - Double exchangeRate; - - if (null == CurrencyFieldType.getCurrency(fromCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); - } - if (null == CurrencyFieldType.getCurrency(toCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); - } - - try { - exchangeRate = Double.parseDouble(rate.getNodeValue()); - } catch (NumberFormatException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); - } - - addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); + if (null == CurrencyFieldType.getCurrency(fromCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); } - } catch (SAXException | XPathExpressionException | ParserConfigurationException | IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error parsing currency config.", e); - } - } catch (IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while opening Currency configuration file "+currencyConfigFile, e); - } finally { - try { - if (is != null) { - is.close(); + if (null == CurrencyFieldType.getCurrency(toCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); + } + + try { + exchangeRate = Double.parseDouble(rate.getNodeValue()); + } catch (NumberFormatException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); } - } catch (IOException e) { - e.printStackTrace(); + + addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); } + } catch (SAXException | IOException | XPathExpressionException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while parsing currency configuration file "+currencyConfigFile, e); } // Atomically swap in the new rates map, if it loaded successfully this.rates = tmpRates;
solr/core/src/java/org/apache/solr/util/SafeXMLParsing.java+120 −0 added@@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.FilterReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.io.StringReader; + +import javax.xml.XMLConstants; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.parsers.ParserConfigurationException; + +import org.apache.commons.io.input.CloseShieldInputStream; +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.solr.common.EmptyEntityResolver; +import org.apache.solr.common.SolrException; +import org.apache.solr.common.util.SuppressForbidden; +import org.apache.solr.common.util.XMLErrorLogger; +import org.slf4j.Logger; +import org.w3c.dom.Document; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; + +/** + * Some utility methods for parsing XML in a safe way. This class can be used to parse XML + * coming from network (completely untrusted) or it can load a config file from a + * {@link ResourceLoader}. In this case it allows external entities and xincludes, but only + * referring to files reachable by the loader. + */ +@SuppressForbidden(reason = "This class uses XML APIs directly that should not be used anywhere else in Solr code") +public final class SafeXMLParsing { + + public static final String SYSTEMID_UNTRUSTED = "untrusted://stream"; + + private SafeXMLParsing() {} + + /** Parses a config file from ResourceLoader. Xinclude and external entities are enabled, but cannot escape the resource loader. */ + public static Document parseConfigXML(Logger log, ResourceLoader loader, String file) throws SAXException, IOException { + try (InputStream in = loader.openResource(file)) { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + try { + dbf.setXIncludeAware(true); + } catch (UnsupportedOperationException e) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); + } + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(new SystemIdResolver(loader)); + db.setErrorHandler(new XMLErrorLogger(log)); + return db.parse(in, SystemIdResolver.createSystemIdFromResourceName(file)); + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given InputStream is not closed. */ + public static Document parseUntrustedXML(Logger log, InputStream in) throws SAXException, IOException { + return getUntrustedDocumentBuilder(log).parse(new CloseShieldInputStream(in), SYSTEMID_UNTRUSTED); + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given Reader is not closed. */ + public static Document parseUntrustedXML(Logger log, Reader reader) throws SAXException, IOException { + final InputSource is = new InputSource(new FilterReader(reader) { + @Override public void close() {} + }); + is.setSystemId(SYSTEMID_UNTRUSTED); + return getUntrustedDocumentBuilder(log).parse(is); + } + + public static Document parseUntrustedXML(Logger log, String xml) throws SAXException, IOException { + return parseUntrustedXML(log, new StringReader(xml)); + } + + private static DocumentBuilder getUntrustedDocumentBuilder(Logger log) { + try { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE); + db.setErrorHandler(new XMLErrorLogger(log)); + return db; + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + private static void trySetDOMFeature(DocumentBuilderFactory factory, String feature, boolean enabled) { + try { + factory.setFeature(feature, enabled); + } catch (Exception ex) { + // ignore + } + } + +}
solr/core/src/test/org/apache/solr/util/TestSafeXMLParsing.java+99 −0 added@@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.lang.invoke.MethodHandles; +import java.nio.charset.StandardCharsets; + +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.lucene.util.LuceneTestCase; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.w3c.dom.Document; + +public class TestSafeXMLParsing extends LuceneTestCase { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + public void testUntrusted() throws Exception { + // TODO: Fix the underlying EmptyEntityResolver to not replace external entities by nothing and instead throw exception: + Document doc = SafeXMLParsing.parseUntrustedXML(log, "<!DOCTYPE test [\n" + + "<!ENTITY internalTerm \"foobar\">\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&internalTerm;&externalTerm;</test>"); + assertEquals("foobar", doc.getDocumentElement().getTextContent()); + } + + InputStream getStringStream(String xml) { + return new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)); + } + + public void testConfig() throws Exception { + final ResourceLoader loader = new ResourceLoader() { + @Override + public InputStream openResource(String resource) throws IOException { + switch (resource) { + case "source1.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source2.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"./include1.xml\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source3.xml": + return getStringStream("<foo xmlns:xi=\"http://www.w3.org/2001/XInclude\">\n" + + " <xi:include href=\"./include2.xml\"/>\n" + + "</foo>"); + case "include1.xml": + return getStringStream("Make XML Great Again!™"); + case "include2.xml": + return getStringStream("<bar>Make XML Great Again!™</bar>"); + } + throw new IOException("Resource not found: " + resource); + } + + @Override + public <T> Class<? extends T> findClass(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + @Override + public <T> T newInstance(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + }; + + IOException ioe = expectThrows(IOException.class, () -> { + SafeXMLParsing.parseConfigXML(log, loader, "source1.xml"); + }); + assertTrue(ioe.getMessage().contains("Cannot resolve absolute systemIDs")); + + Document doc = SafeXMLParsing.parseConfigXML(log, loader, "source2.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent()); + + doc = SafeXMLParsing.parseConfigXML(log, loader, "source3.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent().trim()); + } + +}
d1baf6ba5935SOLR-12450: Don't allow referal to external resources in various config files
4 files changed · +245 −5
solr/CHANGES.txt+19 −0 modified@@ -16,6 +16,25 @@ In this release, there is an example Solr server including a bundled servlet container in the directory named "example". See the Quick Start guide at http://lucene.apache.org/solr/quickstart.html +================== 6.6.5 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.14.v20161028 + +Bug Fixes +---------------------- + + * SOLR-12450: Don't allow referal to external resources in various config files. + (Yuyang Xiao, Uwe Schindler) + ================== 6.6.4 ================== Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
solr/contrib/extraction/src/java/org/apache/solr/handler/extraction/ParseContextConfig.java+7 −5 modified@@ -16,27 +16,31 @@ */ package org.apache.solr.handler.extraction; -import javax.xml.parsers.DocumentBuilderFactory; import java.beans.BeanInfo; import java.beans.Introspector; import java.beans.PropertyDescriptor; import java.beans.PropertyEditor; import java.beans.PropertyEditorManager; -import java.io.InputStream; +import java.lang.invoke.MethodHandles; import java.lang.reflect.Method; import java.util.HashMap; import java.util.Locale; import java.util.Map; import org.apache.solr.core.SolrResourceLoader; +import org.apache.solr.util.SafeXMLParsing; import org.apache.tika.parser.ParseContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.w3c.dom.Document; import org.w3c.dom.Element; import org.w3c.dom.NamedNodeMap; import org.w3c.dom.Node; import org.w3c.dom.NodeList; public class ParseContextConfig { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + private final Map<Class<?>, Object> entries = new HashMap<>(); /** Creates an empty Config without any settings (used as placeholder). */ @@ -54,9 +58,7 @@ public ParseContextConfig(SolrResourceLoader resourceLoader, String parseContext } private static Document loadConfigFile(SolrResourceLoader resourceLoader, String parseContextConfigLoc) throws Exception { - try (InputStream in = resourceLoader.openResource(parseContextConfigLoc)) { - return DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(in, parseContextConfigLoc); - } + return SafeXMLParsing.parseConfigXML(log, resourceLoader, parseContextConfigLoc); } private void extract(Element element, SolrResourceLoader loader) throws Exception {
solr/core/src/java/org/apache/solr/util/SafeXMLParsing.java+120 −0 added@@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.FilterReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.io.StringReader; + +import javax.xml.XMLConstants; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.parsers.ParserConfigurationException; + +import org.apache.commons.io.input.CloseShieldInputStream; +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.solr.common.EmptyEntityResolver; +import org.apache.solr.common.SolrException; +import org.apache.solr.common.util.SuppressForbidden; +import org.apache.solr.common.util.XMLErrorLogger; +import org.slf4j.Logger; +import org.w3c.dom.Document; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; + +/** + * Some utility methods for parsing XML in a safe way. This class can be used to parse XML + * coming from network (completely untrusted) or it can load a config file from a + * {@link ResourceLoader}. In this case it allows external entities and xincludes, but only + * referring to files reachable by the loader. + */ +@SuppressForbidden(reason = "This class uses XML APIs directly that should not be used anywhere else in Solr code") +public final class SafeXMLParsing { + + public static final String SYSTEMID_UNTRUSTED = "untrusted://stream"; + + private SafeXMLParsing() {} + + /** Parses a config file from ResourceLoader. Xinclude and external entities are enabled, but cannot escape the resource loader. */ + public static Document parseConfigXML(Logger log, ResourceLoader loader, String file) throws SAXException, IOException { + try (InputStream in = loader.openResource(file)) { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + try { + dbf.setXIncludeAware(true); + } catch (UnsupportedOperationException e) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); + } + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(new SystemIdResolver(loader)); + db.setErrorHandler(new XMLErrorLogger(log)); + return db.parse(in, SystemIdResolver.createSystemIdFromResourceName(file)); + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given InputStream is not closed. */ + public static Document parseUntrustedXML(Logger log, InputStream in) throws SAXException, IOException { + return getUntrustedDocumentBuilder(log).parse(new CloseShieldInputStream(in), SYSTEMID_UNTRUSTED); + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given Reader is not closed. */ + public static Document parseUntrustedXML(Logger log, Reader reader) throws SAXException, IOException { + final InputSource is = new InputSource(new FilterReader(reader) { + @Override public void close() {} + }); + is.setSystemId(SYSTEMID_UNTRUSTED); + return getUntrustedDocumentBuilder(log).parse(is); + } + + public static Document parseUntrustedXML(Logger log, String xml) throws SAXException, IOException { + return parseUntrustedXML(log, new StringReader(xml)); + } + + private static DocumentBuilder getUntrustedDocumentBuilder(Logger log) { + try { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE); + db.setErrorHandler(new XMLErrorLogger(log)); + return db; + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + private static void trySetDOMFeature(DocumentBuilderFactory factory, String feature, boolean enabled) { + try { + factory.setFeature(feature, enabled); + } catch (Exception ex) { + // ignore + } + } + +}
solr/core/src/test/org/apache/solr/util/TestSafeXMLParsing.java+99 −0 added@@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.lang.invoke.MethodHandles; +import java.nio.charset.StandardCharsets; + +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.lucene.util.LuceneTestCase; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.w3c.dom.Document; + +public class TestSafeXMLParsing extends LuceneTestCase { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + public void testUntrusted() throws Exception { + // TODO: Fix the underlying EmptyEntityResolver to not replace external entities by nothing and instead throw exception: + Document doc = SafeXMLParsing.parseUntrustedXML(log, "<!DOCTYPE test [\n" + + "<!ENTITY internalTerm \"foobar\">\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&internalTerm;&externalTerm;</test>"); + assertEquals("foobar", doc.getDocumentElement().getTextContent()); + } + + InputStream getStringStream(String xml) { + return new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)); + } + + public void testConfig() throws Exception { + final ResourceLoader loader = new ResourceLoader() { + @Override + public InputStream openResource(String resource) throws IOException { + switch (resource) { + case "source1.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source2.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"./include1.xml\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source3.xml": + return getStringStream("<foo xmlns:xi=\"http://www.w3.org/2001/XInclude\">\n" + + " <xi:include href=\"./include2.xml\"/>\n" + + "</foo>"); + case "include1.xml": + return getStringStream("Make XML Great Again!™"); + case "include2.xml": + return getStringStream("<bar>Make XML Great Again!™</bar>"); + } + throw new IOException("Resource not found: " + resource); + } + + @Override + public <T> Class<? extends T> findClass(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + @Override + public <T> T newInstance(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + }; + + IOException ioe = expectThrows(IOException.class, () -> { + SafeXMLParsing.parseConfigXML(log, loader, "source1.xml"); + }); + assertTrue(ioe.getMessage().contains("Cannot resolve absolute systemIDs")); + + Document doc = SafeXMLParsing.parseConfigXML(log, loader, "source2.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent()); + + doc = SafeXMLParsing.parseConfigXML(log, loader, "source3.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent().trim()); + } + +}
e5407c5a9710SOLR-12450: Don't allow referal to external resources in various config files
6 files changed · +313 −128
solr/CHANGES.txt+3 −0 modified@@ -358,6 +358,9 @@ Bug Fixes but 'bin/solr create' tells users to use the default action 'set-property', which fails because the property is not editable. (Steve Rowe) +* SOLR-12450: Don't allow referal to external resources in various config files. + (Yuyang Xiao, Uwe Schindler) + Optimizations ----------------------
solr/contrib/extraction/src/java/org/apache/solr/handler/extraction/ParseContextConfig.java+7 −5 modified@@ -16,27 +16,31 @@ */ package org.apache.solr.handler.extraction; -import javax.xml.parsers.DocumentBuilderFactory; import java.beans.BeanInfo; import java.beans.Introspector; import java.beans.PropertyDescriptor; import java.beans.PropertyEditor; import java.beans.PropertyEditorManager; -import java.io.InputStream; +import java.lang.invoke.MethodHandles; import java.lang.reflect.Method; import java.util.HashMap; import java.util.Locale; import java.util.Map; import org.apache.solr.core.SolrResourceLoader; +import org.apache.solr.util.SafeXMLParsing; import org.apache.tika.parser.ParseContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.w3c.dom.Document; import org.w3c.dom.Element; import org.w3c.dom.NamedNodeMap; import org.w3c.dom.Node; import org.w3c.dom.NodeList; public class ParseContextConfig { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + private final Map<Class<?>, Object> entries = new HashMap<>(); /** Creates an empty Config without any settings (used as placeholder). */ @@ -54,9 +58,7 @@ public ParseContextConfig(SolrResourceLoader resourceLoader, String parseContext } private static Document loadConfigFile(SolrResourceLoader resourceLoader, String parseContextConfigLoc) throws Exception { - try (InputStream in = resourceLoader.openResource(parseContextConfigLoc)) { - return DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(in, parseContextConfigLoc); - } + return SafeXMLParsing.parseConfigXML(log, resourceLoader, parseContextConfigLoc); } private void extract(Element element, SolrResourceLoader loader) throws Exception {
solr/core/src/java/org/apache/solr/schema/AbstractEnumField.java+42 −59 modified@@ -18,14 +18,12 @@ package org.apache.solr.schema; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.Collections; import java.util.HashMap; import java.util.Locale; import java.util.Map; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; + import javax.xml.xpath.XPath; import javax.xml.xpath.XPathConstants; import javax.xml.xpath.XPathExpressionException; @@ -38,8 +36,10 @@ import org.apache.lucene.util.BytesRefBuilder; import org.apache.solr.common.EnumFieldValue; import org.apache.solr.common.SolrException; +import org.apache.solr.core.SolrResourceLoader; import org.apache.solr.response.TextResponseWriter; import org.apache.solr.search.QParser; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -51,7 +51,6 @@ * Abstract Field type for support of string values with custom sort order. */ public abstract class AbstractEnumField extends PrimitiveFieldType { - private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); protected EnumMapping enumMapping; @Override @@ -69,6 +68,8 @@ public EnumMapping getEnumMapping() { * @lucene.internal */ public static final class EnumMapping { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + public static final String PARAM_ENUMS_CONFIG = "enumsConfig"; public static final String PARAM_ENUM_NAME = "enumName"; public static final Integer DEFAULT_VALUE = -1; @@ -105,67 +106,49 @@ public EnumMapping(IndexSchema schema, FieldType fieldType, Map<String, String> ftName + ": No enum name was configured."); } - InputStream is = null; - + final SolrResourceLoader loader = schema.getResourceLoader(); try { - is = schema.getResourceLoader().openResource(enumsConfigFile); - final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - final Document doc = dbf.newDocumentBuilder().parse(is); - final XPathFactory xpathFactory = XPathFactory.newInstance(); - final XPath xpath = xpathFactory.newXPath(); - final String xpathStr = String.format(Locale.ROOT, "/enumsConfig/enum[@name='%s']", enumName); - final NodeList nodes = (NodeList) xpath.evaluate(xpathStr, doc, XPathConstants.NODESET); - final int nodesLength = nodes.getLength(); - if (nodesLength == 0) { - String exceptionMessage = String.format - (Locale.ENGLISH, "%s: No enum configuration found for enum '%s' in %s.", + log.debug("Reloading enums config file from {}", enumsConfigFile); + Document doc = SafeXMLParsing.parseConfigXML(log, loader, enumsConfigFile); + final XPathFactory xpathFactory = XPathFactory.newInstance(); + final XPath xpath = xpathFactory.newXPath(); + final String xpathStr = String.format(Locale.ROOT, "/enumsConfig/enum[@name='%s']", enumName); + final NodeList nodes = (NodeList) xpath.evaluate(xpathStr, doc, XPathConstants.NODESET); + final int nodesLength = nodes.getLength(); + if (nodesLength == 0) { + String exceptionMessage = String.format + (Locale.ENGLISH, "%s: No enum configuration found for enum '%s' in %s.", + ftName, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.NOT_FOUND, exceptionMessage); + } + if (nodesLength > 1) { + if (log.isWarnEnabled()) + log.warn("{}: More than one enum configuration found for enum '{}' in {}. The last one was taken.", + ftName, enumName, enumsConfigFile); + } + final Node enumNode = nodes.item(nodesLength - 1); + final NodeList valueNodes = (NodeList) xpath.evaluate("value", enumNode, XPathConstants.NODESET); + for (int i = 0; i < valueNodes.getLength(); i++) { + final Node valueNode = valueNodes.item(i); + final String valueStr = valueNode.getTextContent(); + if ((valueStr == null) || (valueStr.length() == 0)) { + final String exceptionMessage = String.format + (Locale.ENGLISH, "%s: A value was defined with an no value in enum '%s' in %s.", ftName, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.NOT_FOUND, exceptionMessage); - } - if (nodesLength > 1) { - if (log.isWarnEnabled()) - log.warn("{}: More than one enum configuration found for enum '{}' in {}. The last one was taken.", - ftName, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); } - final Node enumNode = nodes.item(nodesLength - 1); - final NodeList valueNodes = (NodeList) xpath.evaluate("value", enumNode, XPathConstants.NODESET); - for (int i = 0; i < valueNodes.getLength(); i++) { - final Node valueNode = valueNodes.item(i); - final String valueStr = valueNode.getTextContent(); - if ((valueStr == null) || (valueStr.length() == 0)) { - final String exceptionMessage = String.format - (Locale.ENGLISH, "%s: A value was defined with an no value in enum '%s' in %s.", - ftName, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); - } - if (enumStringToIntMap.containsKey(valueStr)) { - final String exceptionMessage = String.format - (Locale.ENGLISH, "%s: A duplicated definition was found for value '%s' in enum '%s' in %s.", - ftName, valueStr, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); - } - enumIntToStringMap.put(i, valueStr); - enumStringToIntMap.put(valueStr, i); + if (enumStringToIntMap.containsKey(valueStr)) { + final String exceptionMessage = String.format + (Locale.ENGLISH, "%s: A duplicated definition was found for value '%s' in enum '%s' in %s.", + ftName, valueStr, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); } + enumIntToStringMap.put(i, valueStr); + enumStringToIntMap.put(valueStr, i); } - catch (ParserConfigurationException | XPathExpressionException | SAXException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, - ftName + ": Error parsing enums config.", e); - } - } - catch (IOException e) { + } catch (IOException | SAXException | XPathExpressionException e) { throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, - ftName + ": Error while opening enums config.", e); - } finally { - try { - if (is != null) { - is.close(); - } - } - catch (IOException e) { - e.printStackTrace(); - } + ftName + ": Error while parsing enums config.", e); } if ((enumStringToIntMap.size() == 0) || (enumIntToStringMap.size() == 0)) {
solr/core/src/java/org/apache/solr/schema/FileExchangeRateProvider.java+42 −64 modified@@ -17,22 +17,21 @@ package org.apache.solr.schema; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; -import javax.xml.xpath.XPath; -import javax.xml.xpath.XPathConstants; -import javax.xml.xpath.XPathExpressionException; -import javax.xml.xpath.XPathFactory; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Set; +import javax.xml.xpath.XPath; +import javax.xml.xpath.XPathConstants; +import javax.xml.xpath.XPathExpressionException; +import javax.xml.xpath.XPathFactory; + import org.apache.lucene.analysis.util.ResourceLoader; import org.apache.solr.common.SolrException; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -46,6 +45,7 @@ */ class FileExchangeRateProvider implements ExchangeRateProvider { private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + protected static final String PARAM_CURRENCY_CONFIG = "currencyConfig"; // Exchange rate map, maps Currency Code -> Currency Code -> Rate @@ -159,71 +159,49 @@ public Set<String> listAvailableCurrencies() { @Override public boolean reload() throws SolrException { - InputStream is = null; Map<String, Map<String, Double>> tmpRates = new HashMap<>(); + log.debug("Reloading exchange rates from file {}", currencyConfigFile); + try { - log.debug("Reloading exchange rates from file "+this.currencyConfigFile); - - is = loader.openResource(currencyConfigFile); - javax.xml.parsers.DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - dbf.setXIncludeAware(true); - dbf.setNamespaceAware(true); - } catch (UnsupportedOperationException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); - } + Document doc = SafeXMLParsing.parseConfigXML(log, loader, currencyConfigFile); + XPathFactory xpathFactory = XPathFactory.newInstance(); + XPath xpath = xpathFactory.newXPath(); + + // Parse exchange rates. + NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); - try { - Document doc = dbf.newDocumentBuilder().parse(is); - XPathFactory xpathFactory = XPathFactory.newInstance(); - XPath xpath = xpathFactory.newXPath(); + for (int i = 0; i < nodes.getLength(); i++) { + Node rateNode = nodes.item(i); + NamedNodeMap attributes = rateNode.getAttributes(); + Node from = attributes.getNamedItem("from"); + Node to = attributes.getNamedItem("to"); + Node rate = attributes.getNamedItem("rate"); + + if (from == null || to == null || rate == null) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); + } - // Parse exchange rates. - NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); + String fromCurrency = from.getNodeValue(); + String toCurrency = to.getNodeValue(); + Double exchangeRate; - for (int i = 0; i < nodes.getLength(); i++) { - Node rateNode = nodes.item(i); - NamedNodeMap attributes = rateNode.getAttributes(); - Node from = attributes.getNamedItem("from"); - Node to = attributes.getNamedItem("to"); - Node rate = attributes.getNamedItem("rate"); - - if (from == null || to == null || rate == null) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); - } - - String fromCurrency = from.getNodeValue(); - String toCurrency = to.getNodeValue(); - Double exchangeRate; - - if (null == CurrencyFieldType.getCurrency(fromCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); - } - if (null == CurrencyFieldType.getCurrency(toCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); - } - - try { - exchangeRate = Double.parseDouble(rate.getNodeValue()); - } catch (NumberFormatException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); - } - - addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); + if (null == CurrencyFieldType.getCurrency(fromCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); } - } catch (SAXException | XPathExpressionException | ParserConfigurationException | IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error parsing currency config.", e); - } - } catch (IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while opening Currency configuration file "+currencyConfigFile, e); - } finally { - try { - if (is != null) { - is.close(); + if (null == CurrencyFieldType.getCurrency(toCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); + } + + try { + exchangeRate = Double.parseDouble(rate.getNodeValue()); + } catch (NumberFormatException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); } - } catch (IOException e) { - e.printStackTrace(); + + addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); } + } catch (SAXException | IOException | XPathExpressionException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while parsing currency configuration file "+currencyConfigFile, e); } // Atomically swap in the new rates map, if it loaded successfully this.rates = tmpRates;
solr/core/src/java/org/apache/solr/util/SafeXMLParsing.java+120 −0 added@@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.FilterReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.io.StringReader; + +import javax.xml.XMLConstants; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.parsers.ParserConfigurationException; + +import org.apache.commons.io.input.CloseShieldInputStream; +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.solr.common.EmptyEntityResolver; +import org.apache.solr.common.SolrException; +import org.apache.solr.common.util.SuppressForbidden; +import org.apache.solr.common.util.XMLErrorLogger; +import org.slf4j.Logger; +import org.w3c.dom.Document; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; + +/** + * Some utility methods for parsing XML in a safe way. This class can be used to parse XML + * coming from network (completely untrusted) or it can load a config file from a + * {@link ResourceLoader}. In this case it allows external entities and xincludes, but only + * referring to files reachable by the loader. + */ +@SuppressForbidden(reason = "This class uses XML APIs directly that should not be used anywhere else in Solr code") +public final class SafeXMLParsing { + + public static final String SYSTEMID_UNTRUSTED = "untrusted://stream"; + + private SafeXMLParsing() {} + + /** Parses a config file from ResourceLoader. Xinclude and external entities are enabled, but cannot escape the resource loader. */ + public static Document parseConfigXML(Logger log, ResourceLoader loader, String file) throws SAXException, IOException { + try (InputStream in = loader.openResource(file)) { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + try { + dbf.setXIncludeAware(true); + } catch (UnsupportedOperationException e) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); + } + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(new SystemIdResolver(loader)); + db.setErrorHandler(new XMLErrorLogger(log)); + return db.parse(in, SystemIdResolver.createSystemIdFromResourceName(file)); + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given InputStream is not closed. */ + public static Document parseUntrustedXML(Logger log, InputStream in) throws SAXException, IOException { + return getUntrustedDocumentBuilder(log).parse(new CloseShieldInputStream(in), SYSTEMID_UNTRUSTED); + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given Reader is not closed. */ + public static Document parseUntrustedXML(Logger log, Reader reader) throws SAXException, IOException { + final InputSource is = new InputSource(new FilterReader(reader) { + @Override public void close() {} + }); + is.setSystemId(SYSTEMID_UNTRUSTED); + return getUntrustedDocumentBuilder(log).parse(is); + } + + public static Document parseUntrustedXML(Logger log, String xml) throws SAXException, IOException { + return parseUntrustedXML(log, new StringReader(xml)); + } + + private static DocumentBuilder getUntrustedDocumentBuilder(Logger log) { + try { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE); + db.setErrorHandler(new XMLErrorLogger(log)); + return db; + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + private static void trySetDOMFeature(DocumentBuilderFactory factory, String feature, boolean enabled) { + try { + factory.setFeature(feature, enabled); + } catch (Exception ex) { + // ignore + } + } + +}
solr/core/src/test/org/apache/solr/util/TestSafeXMLParsing.java+99 −0 added@@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.lang.invoke.MethodHandles; +import java.nio.charset.StandardCharsets; + +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.lucene.util.LuceneTestCase; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.w3c.dom.Document; + +public class TestSafeXMLParsing extends LuceneTestCase { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + public void testUntrusted() throws Exception { + // TODO: Fix the underlying EmptyEntityResolver to not replace external entities by nothing and instead throw exception: + Document doc = SafeXMLParsing.parseUntrustedXML(log, "<!DOCTYPE test [\n" + + "<!ENTITY internalTerm \"foobar\">\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&internalTerm;&externalTerm;</test>"); + assertEquals("foobar", doc.getDocumentElement().getTextContent()); + } + + InputStream getStringStream(String xml) { + return new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)); + } + + public void testConfig() throws Exception { + final ResourceLoader loader = new ResourceLoader() { + @Override + public InputStream openResource(String resource) throws IOException { + switch (resource) { + case "source1.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source2.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"./include1.xml\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source3.xml": + return getStringStream("<foo xmlns:xi=\"http://www.w3.org/2001/XInclude\">\n" + + " <xi:include href=\"./include2.xml\"/>\n" + + "</foo>"); + case "include1.xml": + return getStringStream("Make XML Great Again!™"); + case "include2.xml": + return getStringStream("<bar>Make XML Great Again!™</bar>"); + } + throw new IOException("Resource not found: " + resource); + } + + @Override + public <T> Class<? extends T> findClass(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + @Override + public <T> T newInstance(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + }; + + IOException ioe = expectThrows(IOException.class, () -> { + SafeXMLParsing.parseConfigXML(log, loader, "source1.xml"); + }); + assertTrue(ioe.getMessage().contains("Cannot resolve absolute systemIDs")); + + Document doc = SafeXMLParsing.parseConfigXML(log, loader, "source2.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent()); + + doc = SafeXMLParsing.parseConfigXML(log, loader, "source3.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent().trim()); + } + +}
3aa6086ed99fSOLR-12450: Don't allow referal to external resources in various config files
6 files changed · +313 −128
solr/CHANGES.txt+3 −0 modified@@ -297,6 +297,9 @@ Bug Fixes * SOLR-12416: When creating a time routed alias, the router.autoDeleteAge option wasn't considered. (Joachim Sauer via David Smiley) +* SOLR-12450: Don't allow referal to external resources in various config files. + (Yuyang Xiao, Uwe Schindler) + Optimizations ----------------------
solr/contrib/extraction/src/java/org/apache/solr/handler/extraction/ParseContextConfig.java+7 −5 modified@@ -16,27 +16,31 @@ */ package org.apache.solr.handler.extraction; -import javax.xml.parsers.DocumentBuilderFactory; import java.beans.BeanInfo; import java.beans.Introspector; import java.beans.PropertyDescriptor; import java.beans.PropertyEditor; import java.beans.PropertyEditorManager; -import java.io.InputStream; +import java.lang.invoke.MethodHandles; import java.lang.reflect.Method; import java.util.HashMap; import java.util.Locale; import java.util.Map; import org.apache.solr.core.SolrResourceLoader; +import org.apache.solr.util.SafeXMLParsing; import org.apache.tika.parser.ParseContext; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.w3c.dom.Document; import org.w3c.dom.Element; import org.w3c.dom.NamedNodeMap; import org.w3c.dom.Node; import org.w3c.dom.NodeList; public class ParseContextConfig { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + private final Map<Class<?>, Object> entries = new HashMap<>(); /** Creates an empty Config without any settings (used as placeholder). */ @@ -54,9 +58,7 @@ public ParseContextConfig(SolrResourceLoader resourceLoader, String parseContext } private static Document loadConfigFile(SolrResourceLoader resourceLoader, String parseContextConfigLoc) throws Exception { - try (InputStream in = resourceLoader.openResource(parseContextConfigLoc)) { - return DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(in, parseContextConfigLoc); - } + return SafeXMLParsing.parseConfigXML(log, resourceLoader, parseContextConfigLoc); } private void extract(Element element, SolrResourceLoader loader) throws Exception {
solr/core/src/java/org/apache/solr/schema/AbstractEnumField.java+42 −59 modified@@ -18,14 +18,12 @@ package org.apache.solr.schema; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.Collections; import java.util.HashMap; import java.util.Locale; import java.util.Map; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; + import javax.xml.xpath.XPath; import javax.xml.xpath.XPathConstants; import javax.xml.xpath.XPathExpressionException; @@ -38,8 +36,10 @@ import org.apache.lucene.util.BytesRefBuilder; import org.apache.solr.common.EnumFieldValue; import org.apache.solr.common.SolrException; +import org.apache.solr.core.SolrResourceLoader; import org.apache.solr.response.TextResponseWriter; import org.apache.solr.search.QParser; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -51,7 +51,6 @@ * Abstract Field type for support of string values with custom sort order. */ public abstract class AbstractEnumField extends PrimitiveFieldType { - private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); protected EnumMapping enumMapping; @Override @@ -69,6 +68,8 @@ public EnumMapping getEnumMapping() { * @lucene.internal */ public static final class EnumMapping { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + public static final String PARAM_ENUMS_CONFIG = "enumsConfig"; public static final String PARAM_ENUM_NAME = "enumName"; public static final Integer DEFAULT_VALUE = -1; @@ -105,67 +106,49 @@ public EnumMapping(IndexSchema schema, FieldType fieldType, Map<String, String> ftName + ": No enum name was configured."); } - InputStream is = null; - + final SolrResourceLoader loader = schema.getResourceLoader(); try { - is = schema.getResourceLoader().openResource(enumsConfigFile); - final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - final Document doc = dbf.newDocumentBuilder().parse(is); - final XPathFactory xpathFactory = XPathFactory.newInstance(); - final XPath xpath = xpathFactory.newXPath(); - final String xpathStr = String.format(Locale.ROOT, "/enumsConfig/enum[@name='%s']", enumName); - final NodeList nodes = (NodeList) xpath.evaluate(xpathStr, doc, XPathConstants.NODESET); - final int nodesLength = nodes.getLength(); - if (nodesLength == 0) { - String exceptionMessage = String.format - (Locale.ENGLISH, "%s: No enum configuration found for enum '%s' in %s.", + log.debug("Reloading enums config file from {}", enumsConfigFile); + Document doc = SafeXMLParsing.parseConfigXML(log, loader, enumsConfigFile); + final XPathFactory xpathFactory = XPathFactory.newInstance(); + final XPath xpath = xpathFactory.newXPath(); + final String xpathStr = String.format(Locale.ROOT, "/enumsConfig/enum[@name='%s']", enumName); + final NodeList nodes = (NodeList) xpath.evaluate(xpathStr, doc, XPathConstants.NODESET); + final int nodesLength = nodes.getLength(); + if (nodesLength == 0) { + String exceptionMessage = String.format + (Locale.ENGLISH, "%s: No enum configuration found for enum '%s' in %s.", + ftName, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.NOT_FOUND, exceptionMessage); + } + if (nodesLength > 1) { + if (log.isWarnEnabled()) + log.warn("{}: More than one enum configuration found for enum '{}' in {}. The last one was taken.", + ftName, enumName, enumsConfigFile); + } + final Node enumNode = nodes.item(nodesLength - 1); + final NodeList valueNodes = (NodeList) xpath.evaluate("value", enumNode, XPathConstants.NODESET); + for (int i = 0; i < valueNodes.getLength(); i++) { + final Node valueNode = valueNodes.item(i); + final String valueStr = valueNode.getTextContent(); + if ((valueStr == null) || (valueStr.length() == 0)) { + final String exceptionMessage = String.format + (Locale.ENGLISH, "%s: A value was defined with an no value in enum '%s' in %s.", ftName, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.NOT_FOUND, exceptionMessage); - } - if (nodesLength > 1) { - if (log.isWarnEnabled()) - log.warn("{}: More than one enum configuration found for enum '{}' in {}. The last one was taken.", - ftName, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); } - final Node enumNode = nodes.item(nodesLength - 1); - final NodeList valueNodes = (NodeList) xpath.evaluate("value", enumNode, XPathConstants.NODESET); - for (int i = 0; i < valueNodes.getLength(); i++) { - final Node valueNode = valueNodes.item(i); - final String valueStr = valueNode.getTextContent(); - if ((valueStr == null) || (valueStr.length() == 0)) { - final String exceptionMessage = String.format - (Locale.ENGLISH, "%s: A value was defined with an no value in enum '%s' in %s.", - ftName, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); - } - if (enumStringToIntMap.containsKey(valueStr)) { - final String exceptionMessage = String.format - (Locale.ENGLISH, "%s: A duplicated definition was found for value '%s' in enum '%s' in %s.", - ftName, valueStr, enumName, enumsConfigFile); - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); - } - enumIntToStringMap.put(i, valueStr); - enumStringToIntMap.put(valueStr, i); + if (enumStringToIntMap.containsKey(valueStr)) { + final String exceptionMessage = String.format + (Locale.ENGLISH, "%s: A duplicated definition was found for value '%s' in enum '%s' in %s.", + ftName, valueStr, enumName, enumsConfigFile); + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, exceptionMessage); } + enumIntToStringMap.put(i, valueStr); + enumStringToIntMap.put(valueStr, i); } - catch (ParserConfigurationException | XPathExpressionException | SAXException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, - ftName + ": Error parsing enums config.", e); - } - } - catch (IOException e) { + } catch (IOException | SAXException | XPathExpressionException e) { throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, - ftName + ": Error while opening enums config.", e); - } finally { - try { - if (is != null) { - is.close(); - } - } - catch (IOException e) { - e.printStackTrace(); - } + ftName + ": Error while parsing enums config.", e); } if ((enumStringToIntMap.size() == 0) || (enumIntToStringMap.size() == 0)) {
solr/core/src/java/org/apache/solr/schema/FileExchangeRateProvider.java+42 −64 modified@@ -17,22 +17,21 @@ package org.apache.solr.schema; -import javax.xml.parsers.DocumentBuilderFactory; -import javax.xml.parsers.ParserConfigurationException; -import javax.xml.xpath.XPath; -import javax.xml.xpath.XPathConstants; -import javax.xml.xpath.XPathExpressionException; -import javax.xml.xpath.XPathFactory; import java.io.IOException; -import java.io.InputStream; import java.lang.invoke.MethodHandles; import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Set; +import javax.xml.xpath.XPath; +import javax.xml.xpath.XPathConstants; +import javax.xml.xpath.XPathExpressionException; +import javax.xml.xpath.XPathFactory; + import org.apache.lucene.analysis.util.ResourceLoader; import org.apache.solr.common.SolrException; +import org.apache.solr.util.SafeXMLParsing; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.w3c.dom.Document; @@ -46,6 +45,7 @@ */ class FileExchangeRateProvider implements ExchangeRateProvider { private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + protected static final String PARAM_CURRENCY_CONFIG = "currencyConfig"; // Exchange rate map, maps Currency Code -> Currency Code -> Rate @@ -159,71 +159,49 @@ public Set<String> listAvailableCurrencies() { @Override public boolean reload() throws SolrException { - InputStream is = null; Map<String, Map<String, Double>> tmpRates = new HashMap<>(); + log.debug("Reloading exchange rates from file {}", currencyConfigFile); + try { - log.debug("Reloading exchange rates from file "+this.currencyConfigFile); - - is = loader.openResource(currencyConfigFile); - javax.xml.parsers.DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); - try { - dbf.setXIncludeAware(true); - dbf.setNamespaceAware(true); - } catch (UnsupportedOperationException e) { - throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); - } + Document doc = SafeXMLParsing.parseConfigXML(log, loader, currencyConfigFile); + XPathFactory xpathFactory = XPathFactory.newInstance(); + XPath xpath = xpathFactory.newXPath(); + + // Parse exchange rates. + NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); - try { - Document doc = dbf.newDocumentBuilder().parse(is); - XPathFactory xpathFactory = XPathFactory.newInstance(); - XPath xpath = xpathFactory.newXPath(); + for (int i = 0; i < nodes.getLength(); i++) { + Node rateNode = nodes.item(i); + NamedNodeMap attributes = rateNode.getAttributes(); + Node from = attributes.getNamedItem("from"); + Node to = attributes.getNamedItem("to"); + Node rate = attributes.getNamedItem("rate"); + + if (from == null || to == null || rate == null) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); + } - // Parse exchange rates. - NodeList nodes = (NodeList) xpath.evaluate("/currencyConfig/rates/rate", doc, XPathConstants.NODESET); + String fromCurrency = from.getNodeValue(); + String toCurrency = to.getNodeValue(); + Double exchangeRate; - for (int i = 0; i < nodes.getLength(); i++) { - Node rateNode = nodes.item(i); - NamedNodeMap attributes = rateNode.getAttributes(); - Node from = attributes.getNamedItem("from"); - Node to = attributes.getNamedItem("to"); - Node rate = attributes.getNamedItem("rate"); - - if (from == null || to == null || rate == null) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Exchange rate missing attributes (required: from, to, rate) " + rateNode); - } - - String fromCurrency = from.getNodeValue(); - String toCurrency = to.getNodeValue(); - Double exchangeRate; - - if (null == CurrencyFieldType.getCurrency(fromCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); - } - if (null == CurrencyFieldType.getCurrency(toCurrency)) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); - } - - try { - exchangeRate = Double.parseDouble(rate.getNodeValue()); - } catch (NumberFormatException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); - } - - addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); + if (null == CurrencyFieldType.getCurrency(fromCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'from' currency not supported in this JVM: " + fromCurrency); } - } catch (SAXException | XPathExpressionException | ParserConfigurationException | IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error parsing currency config.", e); - } - } catch (IOException e) { - throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while opening Currency configuration file "+currencyConfigFile, e); - } finally { - try { - if (is != null) { - is.close(); + if (null == CurrencyFieldType.getCurrency(toCurrency)) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Specified 'to' currency not supported in this JVM: " + toCurrency); + } + + try { + exchangeRate = Double.parseDouble(rate.getNodeValue()); + } catch (NumberFormatException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could not parse exchange rate: " + rateNode, e); } - } catch (IOException e) { - e.printStackTrace(); + + addRate(tmpRates, fromCurrency, toCurrency, exchangeRate); } + } catch (SAXException | IOException | XPathExpressionException e) { + throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Error while parsing currency configuration file "+currencyConfigFile, e); } // Atomically swap in the new rates map, if it loaded successfully this.rates = tmpRates;
solr/core/src/java/org/apache/solr/util/SafeXMLParsing.java+120 −0 added@@ -0,0 +1,120 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.FilterReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.io.StringReader; + +import javax.xml.XMLConstants; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import javax.xml.parsers.ParserConfigurationException; + +import org.apache.commons.io.input.CloseShieldInputStream; +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.solr.common.EmptyEntityResolver; +import org.apache.solr.common.SolrException; +import org.apache.solr.common.util.SuppressForbidden; +import org.apache.solr.common.util.XMLErrorLogger; +import org.slf4j.Logger; +import org.w3c.dom.Document; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; + +/** + * Some utility methods for parsing XML in a safe way. This class can be used to parse XML + * coming from network (completely untrusted) or it can load a config file from a + * {@link ResourceLoader}. In this case it allows external entities and xincludes, but only + * referring to files reachable by the loader. + */ +@SuppressForbidden(reason = "This class uses XML APIs directly that should not be used anywhere else in Solr code") +public final class SafeXMLParsing { + + public static final String SYSTEMID_UNTRUSTED = "untrusted://stream"; + + private SafeXMLParsing() {} + + /** Parses a config file from ResourceLoader. Xinclude and external entities are enabled, but cannot escape the resource loader. */ + public static Document parseConfigXML(Logger log, ResourceLoader loader, String file) throws SAXException, IOException { + try (InputStream in = loader.openResource(file)) { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + try { + dbf.setXIncludeAware(true); + } catch (UnsupportedOperationException e) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser doesn't support XInclude option", e); + } + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(new SystemIdResolver(loader)); + db.setErrorHandler(new XMLErrorLogger(log)); + return db.parse(in, SystemIdResolver.createSystemIdFromResourceName(file)); + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given InputStream is not closed. */ + public static Document parseUntrustedXML(Logger log, InputStream in) throws SAXException, IOException { + return getUntrustedDocumentBuilder(log).parse(new CloseShieldInputStream(in), SYSTEMID_UNTRUSTED); + } + + /** Parses the given InputStream as XML, disabling any external entities with secure processing enabled. + * The given Reader is not closed. */ + public static Document parseUntrustedXML(Logger log, Reader reader) throws SAXException, IOException { + final InputSource is = new InputSource(new FilterReader(reader) { + @Override public void close() {} + }); + is.setSystemId(SYSTEMID_UNTRUSTED); + return getUntrustedDocumentBuilder(log).parse(is); + } + + public static Document parseUntrustedXML(Logger log, String xml) throws SAXException, IOException { + return parseUntrustedXML(log, new StringReader(xml)); + } + + private static DocumentBuilder getUntrustedDocumentBuilder(Logger log) { + try { + final DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); + dbf.setValidating(false); + dbf.setNamespaceAware(true); + trySetDOMFeature(dbf, XMLConstants.FEATURE_SECURE_PROCESSING, true); + + final DocumentBuilder db = dbf.newDocumentBuilder(); + db.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE); + db.setErrorHandler(new XMLErrorLogger(log)); + return db; + } catch (ParserConfigurationException pce) { + throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "XML parser cannot be configured", pce); + } + } + + private static void trySetDOMFeature(DocumentBuilderFactory factory, String feature, boolean enabled) { + try { + factory.setFeature(feature, enabled); + } catch (Exception ex) { + // ignore + } + } + +}
solr/core/src/test/org/apache/solr/util/TestSafeXMLParsing.java+99 −0 added@@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.util; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.lang.invoke.MethodHandles; +import java.nio.charset.StandardCharsets; + +import org.apache.lucene.analysis.util.ResourceLoader; +import org.apache.lucene.util.LuceneTestCase; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.w3c.dom.Document; + +public class TestSafeXMLParsing extends LuceneTestCase { + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + + public void testUntrusted() throws Exception { + // TODO: Fix the underlying EmptyEntityResolver to not replace external entities by nothing and instead throw exception: + Document doc = SafeXMLParsing.parseUntrustedXML(log, "<!DOCTYPE test [\n" + + "<!ENTITY internalTerm \"foobar\">\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&internalTerm;&externalTerm;</test>"); + assertEquals("foobar", doc.getDocumentElement().getTextContent()); + } + + InputStream getStringStream(String xml) { + return new ByteArrayInputStream(xml.getBytes(StandardCharsets.UTF_8)); + } + + public void testConfig() throws Exception { + final ResourceLoader loader = new ResourceLoader() { + @Override + public InputStream openResource(String resource) throws IOException { + switch (resource) { + case "source1.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"foo://bar.xyz/external\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source2.xml": + return getStringStream("<!DOCTYPE test [\n" + + "<!ENTITY externalTerm SYSTEM \"./include1.xml\">\n" + + "]>\n" + + "<test>&externalTerm;</test>"); + case "source3.xml": + return getStringStream("<foo xmlns:xi=\"http://www.w3.org/2001/XInclude\">\n" + + " <xi:include href=\"./include2.xml\"/>\n" + + "</foo>"); + case "include1.xml": + return getStringStream("Make XML Great Again!™"); + case "include2.xml": + return getStringStream("<bar>Make XML Great Again!™</bar>"); + } + throw new IOException("Resource not found: " + resource); + } + + @Override + public <T> Class<? extends T> findClass(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + @Override + public <T> T newInstance(String cname, Class<T> expectedType) { + throw new UnsupportedOperationException(); + } + + }; + + IOException ioe = expectThrows(IOException.class, () -> { + SafeXMLParsing.parseConfigXML(log, loader, "source1.xml"); + }); + assertTrue(ioe.getMessage().contains("Cannot resolve absolute systemIDs")); + + Document doc = SafeXMLParsing.parseConfigXML(log, loader, "source2.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent()); + + doc = SafeXMLParsing.parseConfigXML(log, loader, "source3.xml"); + assertEquals("Make XML Great Again!™", doc.getDocumentElement().getTextContent().trim()); + } + +}
Vulnerability mechanics
Generated on May 9, 2026. Inputs: CWE entries + fix-commit diffs from this CVE's patches. Citations validated against bundle.
References
11- github.com/advisories/GHSA-7px3-6f6g-hxcjghsaADVISORY
- nvd.nist.gov/vuln/detail/CVE-2018-8026ghsaADVISORY
- www.securityfocus.com/bid/104690mitrevdb-entryx_refsource_BID
- github.com/apache/lucene-solr/commit/1880d4824e6c5f98170b9a00aad1d437ee2aa12ghsaWEB
- github.com/apache/lucene-solr/commit/3aa6086ed99fa7158d423dc7c33dae6da466b09ghsaWEB
- github.com/apache/lucene-solr/commit/d1baf6ba593561f39e2da0a71a8440797005b55ghsaWEB
- github.com/apache/lucene-solr/commit/e5407c5a9710247e5f728aae36224a245a51f0bghsaWEB
- issues.apache.org/jira/browse/SOLR-12450ghsax_refsource_CONFIRMWEB
- mail-archives.apache.org/mod_mbox/lucene-solr-user/201807.mbox/%3C0cdc01d413b7%24f97ba580%24ec72f080%24%40apache.org%3Eghsamailing-listx_refsource_MLISTWEB
- security.netapp.com/advisory/ntap-20190307-0002ghsaWEB
- security.netapp.com/advisory/ntap-20190307-0002/mitrex_refsource_CONFIRM
News mentions
0No linked articles in our index yet.