Denial of Service in Tensorflow
Description
In Tensorflow before versions 1.15.4, 2.0.3, 2.1.2, 2.2.1 and 2.3.1, changing the TensorFlow's SavedModel protocol buffer and altering the name of required keys results in segfaults and data corruption while loading the model. This can cause a denial of service in products using tensorflow-serving or other inference-as-a-service installments. Fixed were added in commits f760f88b4267d981e13f4b302c437ae800445968 and fcfef195637c6e365577829c4d67681695956e7d (both going into TensorFlow 2.2.0 and 2.3.0 but not yet backported to earlier versions). However, this was not enough, as #41097 reports a different failure mode. The issue is patched in commit adf095206f25471e864a8e63a0f1caef53a0e3a6, and is released in TensorFlow versions 1.15.4, 2.0.3, 2.1.2, 2.2.1, or 2.3.1.
Affected packages
Versions sourced from the GitHub Security Advisory.
| Package | Affected versions | Patched versions |
|---|---|---|
tensorflowPyPI | < 1.15.4 | 1.15.4 |
tensorflowPyPI | >= 2.0.0, < 2.0.3 | 2.0.3 |
tensorflowPyPI | >= 2.1.0, < 2.1.2 | 2.1.2 |
tensorflowPyPI | >= 2.2.0, < 2.2.1 | 2.2.1 |
tensorflowPyPI | >= 2.3.0, < 2.3.1 | 2.3.1 |
tensorflow-cpuPyPI | < 1.15.4 | 1.15.4 |
tensorflow-cpuPyPI | >= 2.0.0, < 2.0.3 | 2.0.3 |
tensorflow-cpuPyPI | >= 2.1.0, < 2.1.2 | 2.1.2 |
tensorflow-cpuPyPI | >= 2.2.0, < 2.2.1 | 2.2.1 |
tensorflow-cpuPyPI | >= 2.3.0, < 2.3.1 | 2.3.1 |
tensorflow-gpuPyPI | < 1.15.4 | 1.15.4 |
tensorflow-gpuPyPI | >= 2.0.0, < 2.0.3 | 2.0.3 |
tensorflow-gpuPyPI | >= 2.1.0, < 2.1.2 | 2.1.2 |
tensorflow-gpuPyPI | >= 2.2.0, < 2.2.1 | 2.2.1 |
tensorflow-gpuPyPI | >= 2.3.0, < 2.3.1 | 2.3.1 |
Affected products
1- Range: < 1.15.4
Patches
3adf095206f25Validate `NodeDef`s from `FunctionDefLibrary` of a `GraphDef`.
6 files changed · +48 −15
tensorflow/cc/saved_model/loader.cc+31 −15 modified@@ -21,6 +21,7 @@ limitations under the License. #include "tensorflow/cc/saved_model/loader_util.h" #include "tensorflow/cc/saved_model/reader.h" #include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/function.pb.h" #include "tensorflow/core/framework/node_def.pb.h" #include "tensorflow/core/framework/tensor.pb.h" #include "tensorflow/core/lib/io/path.h" @@ -73,26 +74,41 @@ uint64 GetLatencyMicroseconds(const uint64 start_microseconds) { // Ensure that constant tensors loaded from the saved model have valid shape. // Also ensure that constant nodes have a value assigned to them. // TODO(b/154763635): this is temporary and will be replaced with a better audit +static Status ValidateNode(const NodeDef& node) { + const auto node_iterator = node.attr().find("value"); + if (node_iterator != node.attr().end()) { + AttrValue node_value = node_iterator->second; + if (node_value.has_tensor()) { + const PartialTensorShape node_shape(node_value.tensor().tensor_shape()); + if (node_shape.num_elements() < 0) { + return errors::FailedPrecondition( + "Saved model contains node \"", node.name(), "\" (op \"", node.op(), + "\") which initializes from a tensor with ", + node_shape.num_elements(), " elements"); + } + } + } else if (node.op() == "Const") { + return errors::FailedPrecondition( + "Saved model contains node \"", node.name(), + "\" which is a constant tensor but no value has been provided"); + } + return Status::OK(); +} + static Status ValidateSavedTensors(const GraphDef& graph_def) { for (const auto& node : graph_def.node()) { - const auto node_iterator = node.attr().find("value"); - if (node_iterator != node.attr().end()) { - AttrValue node_value = node_iterator->second; - if (node_value.has_tensor()) { - const PartialTensorShape node_shape(node_value.tensor().tensor_shape()); - if (node_shape.num_elements() < 0) { - return errors::FailedPrecondition( - "Saved model contains node \"", node.name(), "\" (op \"", - node.op(), "\") which initializes from a tensor with ", - node_shape.num_elements(), " elements"); - } + TF_RETURN_IF_ERROR(ValidateNode(node)); + } + + if (graph_def.has_library()) { + const FunctionDefLibrary& library = graph_def.library(); + for (const auto& function : library.function()) { + for (const auto& node : function.node_def()) { + TF_RETURN_IF_ERROR(ValidateNode(node)); } - } else if (node.op() == "Const") { - return errors::FailedPrecondition( - "Saved model contains node \"", node.name(), - "\" which is a constant tensor but no value has been provided"); } } + return Status::OK(); }
tensorflow/cc/saved_model/saved_model_bundle_test.cc+17 −0 modified@@ -45,6 +45,8 @@ constexpr char kTestFuzzGeneratedNegativeShape[] = "cc/saved_model/testdata/fuzz_generated/negative_shape"; constexpr char kTestFuzzGeneratedConstWithNoValue[] = "cc/saved_model/testdata/fuzz_generated/const_with_no_value"; +constexpr char kTestFuzzGeneratedBadNodeAttr[] = + "cc/saved_model/testdata/fuzz_generated/bad_node_attr"; class LoaderTest : public ::testing::Test { protected: @@ -328,5 +330,20 @@ TEST_F(LoaderTest, ConstNoValue) { std::string::npos); } +TEST_F(LoaderTest, BadNodeAttr) { + SavedModelBundle bundle; + RunOptions run_options; + SessionOptions session_options; + + const string export_dir = + io::JoinPath(testing::TensorFlowSrcRoot(), kTestFuzzGeneratedBadNodeAttr); + Status st = LoadSavedModel(session_options, run_options, export_dir, + {kSavedModelTagServe}, &bundle); + EXPECT_FALSE(st.ok()); + EXPECT_NE( + st.error_message().find("constant tensor but no value has been provided"), + std::string::npos); +} + } // namespace } // namespace tensorflow
tensorflow/cc/saved_model/testdata/fuzz_generated/bad_node_attr/assets/empty+0 −0 addedtensorflow/cc/saved_model/testdata/fuzz_generated/bad_node_attr/saved_model.pb+0 −0 addedtensorflow/cc/saved_model/testdata/fuzz_generated/bad_node_attr/variables/variables.data-00000-of-00001+0 −0 addedtensorflow/cc/saved_model/testdata/fuzz_generated/bad_node_attr/variables/variables.index+0 −0 added
fcfef195637cPrevent loading saved models where constant nodes have no tensor value.
4 files changed · +22 −3
tensorflow/cc/saved_model/loader.cc+5 −0 modified@@ -70,6 +70,7 @@ uint64 GetLatencyMicroseconds(const uint64 start_microseconds) { } // Ensure that constant tensors loaded from the saved model have valid shape. +// Also ensure that constant nodes have a value assigned to them. // TODO(b/154763635): this is temporary and will be replaced with a better audit static Status ValidateSavedTensors(const GraphDef& graph_def) { for (const auto& node : graph_def.node()) { @@ -85,6 +86,10 @@ static Status ValidateSavedTensors(const GraphDef& graph_def) { node_shape.num_elements(), " elements"); } } + } else if (node.op() == "Const") { + return errors::FailedPrecondition( + "Saved model contains node \"", node.name(), + "\" which is a constant tensor but no value has been provided"); } } return Status::OK();
tensorflow/cc/saved_model/saved_model_bundle_test.cc+17 −3 modified@@ -40,8 +40,10 @@ constexpr char kTestDataInitOpV2[] = "cc/saved_model/testdata/half_plus_two_v2/00000123"; constexpr char kTestDataV2DebugInfo[] = "cc/saved_model/testdata/x_plus_y_v2_debuginfo"; -constexpr char kTestNegativeShapeFuzzGenerated[] = - "cc/saved_model/testdata/negative_shape/fuzz_generated"; +constexpr char kTestFuzzGeneratedNegativeShape[] = + "cc/saved_model/testdata/fuzz_generated/negative_shape"; +constexpr char kTestFuzzGeneratedConstWithNoValue[] = + "cc/saved_model/testdata/fuzz_generated/const_with_no_value"; class LoaderTest : public ::testing::Test { protected: @@ -264,7 +266,19 @@ TEST_F(LoaderTest, NegativeShapeDimension) { SessionOptions session_options; const string export_dir = io::JoinPath(testing::TensorFlowSrcRoot(), - kTestNegativeShapeFuzzGenerated); + kTestFuzzGeneratedNegativeShape); + Status st = LoadSavedModel(session_options, run_options, export_dir, + {kSavedModelTagServe}, &bundle); + EXPECT_FALSE(st.ok()); +} + +TEST_F(LoaderTest, ConstNoValue) { + SavedModelBundle bundle; + RunOptions run_options; + SessionOptions session_options; + + const string export_dir = io::JoinPath(testing::TensorFlowSrcRoot(), + kTestFuzzGeneratedConstWithNoValue); Status st = LoadSavedModel(session_options, run_options, export_dir, {kSavedModelTagServe}, &bundle); EXPECT_FALSE(st.ok());
tensorflow/cc/saved_model/testdata/fuzz_generated/const_with_no_value+0 −0 addedtensorflow/cc/saved_model/testdata/fuzz_generated/negative_shape+0 −0 renamed
f760f88b4267Properly handle negative shape dimensions from improper saved models.
3 files changed · +40 −0
tensorflow/cc/saved_model/loader.cc+26 −0 modified@@ -19,12 +19,16 @@ limitations under the License. #include "tensorflow/cc/saved_model/constants.h" #include "tensorflow/cc/saved_model/reader.h" +#include "tensorflow/core/framework/attr_value.pb.h" +#include "tensorflow/core/framework/node_def.pb.h" +#include "tensorflow/core/framework/tensor.pb.h" #include "tensorflow/core/lib/io/path.h" #include "tensorflow/core/lib/monitoring/counter.h" #include "tensorflow/core/lib/monitoring/sampler.h" #include "tensorflow/core/lib/strings/str_util.h" #include "tensorflow/core/lib/strings/strcat.h" #include "tensorflow/core/platform/env.h" +#include "tensorflow/core/platform/errors.h" #include "tensorflow/core/platform/protobuf_internal.h" #include "tensorflow/core/protobuf/graph_debug_info.pb.h" #include "tensorflow/core/protobuf/saver.pb.h" @@ -65,12 +69,34 @@ uint64 GetLatencyMicroseconds(const uint64 start_microseconds) { return end_microseconds - start_microseconds; } +// Ensure that constant tensors loaded from the saved model have valid shape. +// TODO(b/154763635): this is temporary and will be replaced with a better audit +static Status ValidateSavedTensors(const GraphDef& graph_def) { + for (const auto& node : graph_def.node()) { + const auto node_iterator = node.attr().find("value"); + if (node_iterator != node.attr().end()) { + AttrValue node_value = node_iterator->second; + if (node_value.has_tensor()) { + const PartialTensorShape node_shape(node_value.tensor().tensor_shape()); + if (node_shape.num_elements() < 0) { + return errors::FailedPrecondition( + "Saved model contains node \"", node.name(), "\" (op \"", + node.op(), "\") which initializes from a tensor with ", + node_shape.num_elements(), " elements"); + } + } + } + } + return Status::OK(); +} + Status LoadMetaGraphIntoSession(const MetaGraphDef& meta_graph_def, const SessionOptions& session_options, std::unique_ptr<Session>* session) { Session* session_p = nullptr; TF_RETURN_IF_ERROR(NewSession(session_options, &session_p)); session->reset(session_p); + TF_RETURN_IF_ERROR(ValidateSavedTensors(meta_graph_def.graph_def())); return (*session)->Create(meta_graph_def.graph_def()); }
tensorflow/cc/saved_model/saved_model_bundle_test.cc+14 −0 modified@@ -40,6 +40,8 @@ constexpr char kTestDataInitOpV2[] = "cc/saved_model/testdata/half_plus_two_v2/00000123"; constexpr char kTestDataV2DebugInfo[] = "cc/saved_model/testdata/x_plus_y_v2_debuginfo"; +constexpr char kTestNegativeShapeFuzzGenerated[] = + "cc/saved_model/testdata/negative_shape/fuzz_generated"; class LoaderTest : public ::testing::Test { protected: @@ -256,5 +258,17 @@ TEST_F(LoaderTest, SavedModelV2DebugInfo) { EXPECT_NE(bundle.debug_info.get(), nullptr); } +TEST_F(LoaderTest, NegativeShapeDimension) { + SavedModelBundle bundle; + RunOptions run_options; + SessionOptions session_options; + + const string export_dir = io::JoinPath(testing::TensorFlowSrcRoot(), + kTestNegativeShapeFuzzGenerated); + Status st = LoadSavedModel(session_options, run_options, export_dir, + {kSavedModelTagServe}, &bundle); + EXPECT_FALSE(st.ok()); +} + } // namespace } // namespace tensorflow
tensorflow/cc/saved_model/testdata/negative_shape/fuzz_generated+0 −0 added
Vulnerability mechanics
Generated by null/stub on May 9, 2026. Inputs: CWE entries + fix-commit diffs from this CVE's patches. Citations validated against bundle.
References
11- lists.opensuse.org/opensuse-security-announce/2020-10/msg00065.htmlghsavendor-advisoryx_refsource_SUSEWEB
- github.com/advisories/GHSA-w5gh-2wr2-pm6gghsaADVISORY
- nvd.nist.gov/vuln/detail/CVE-2020-15206ghsaADVISORY
- github.com/pypa/advisory-database/tree/main/vulns/tensorflow-cpu/PYSEC-2020-286.yamlghsaWEB
- github.com/pypa/advisory-database/tree/main/vulns/tensorflow-gpu/PYSEC-2020-321.yamlghsaWEB
- github.com/pypa/advisory-database/tree/main/vulns/tensorflow/PYSEC-2020-129.yamlghsaWEB
- github.com/tensorflow/tensorflow/commit/adf095206f25471e864a8e63a0f1caef53a0e3a6ghsax_refsource_MISCWEB
- github.com/tensorflow/tensorflow/commit/f760f88b4267d981e13f4b302c437ae800445968ghsaWEB
- github.com/tensorflow/tensorflow/commit/fcfef195637c6e365577829c4d67681695956e7dghsaWEB
- github.com/tensorflow/tensorflow/releases/tag/v2.3.1ghsax_refsource_MISCWEB
- github.com/tensorflow/tensorflow/security/advisories/GHSA-w5gh-2wr2-pm6gghsax_refsource_CONFIRMWEB
News mentions
0No linked articles in our index yet.