VYPR
Moderate severityNVD Advisory· Published Apr 25, 2024· Updated Aug 2, 2024

vyper default functions don't respect nonreentrancy keys

CVE-2024-32648

Description

Vyper is a pythonic Smart Contract Language for the Ethereum virtual machine. Prior to version 0.3.0, default functions don't respect nonreentrancy keys and the lock isn't emitted. No vulnerable production contracts were found. Additionally, using a lock on a default function is a very sparsely used pattern. As such, the impact is low. Version 0.3.0 contains a patch for the issue.

AI Insight

LLM-synthesized narrative grounded in this CVE's description and references.

Vyper's default functions prior to 0.3.0 fail to respect nonreentrancy keys, allowing potential reentrancy, but no production contracts affected and usage is sparse.

Vulnerability

CVE-2024-32648 affects Vyper, a Pythonic smart contract language for the Ethereum Virtual Machine. In versions prior to 0.3.0, default functions (i.e., __default__) do not properly enforce nonreentrancy keys and the reentrancy lock is not emitted. This means that if a contract uses a reentrancy lock on a default function, the lock may not actually prevent reentrant calls [1].

Exploitation

To exploit this issue, an attacker would need to interact with a contract that uses a default function with a nonreentrancy key. The attacker could then call the default function recursively, potentially bypassing the intended protection. However, the use of reentrancy locks on default functions is extremely rare in practice, and no known production contracts are affected [1].

Impact

The overall impact is considered low because the vulnerable pattern is seldom used and no real-world exploitation has been observed. If a contract were vulnerable, an attacker could potentially reenter the default function and manipulate state in unexpected ways, but the practical risk is minimal given the lack of deployed instances [1].

Mitigation

The issue is patched in Vyper version 0.3.0. Developers using Vyper should upgrade to 0.3.0 or later to ensure that nonreentrancy locks are properly enforced on default functions [1].

AI Insight generated on May 20, 2026. Synthesized from this CVE's description and the cited reference URLs; citations are validated against the source bundle.

Affected packages

Versions sourced from the GitHub Security Advisory.

PackageAffected versionsPatched versions
vyperPyPI
< 0.3.00.3.0

Affected products

1

Patches

1
93287e5ac184

Merge pull request #2447 from charles-cooper/abi_refactor

https://github.com/vyperlang/vyperCharles CooperOct 3, 2021via ghsa
56 files changed · +1832 2430
  • docs/built-in-functions.rst+1 1 modified
    @@ -631,7 +631,7 @@ Utilities
         Once this function has seen more use we provisionally plan to put it into the ``ethereum.abi`` namespace.
     
         * ``*args``: Arbitrary arguments
    -    * ``ensure_tuple``: If set to True, ensures that even a single argument is encoded as a tuple. In other words, ``bytes`` gets encoded as ``(bytes,)``. This is the calling convention for Vyper and Solidity functions. Except for very specific use cases, this should be set to True. Must be a literal.
    +    * ``ensure_tuple``: If set to True, ensures that even a single argument is encoded as a tuple. In other words, ``bytes`` gets encoded as ``(bytes,)``, and ``(bytes,)`` gets encoded as ``((bytes,),)`` This is the calling convention for Vyper and Solidity functions. Except for very specific use cases, this should be set to True. Must be a literal.
         * ``method_id``: A literal hex or Bytes[4] value to append to the beginning of the abi-encoded bytestring.
     
         Returns a bytestring whose max length is determined by the arguments. For example, encoding a ``Bytes[32]`` results in a ``Bytes[64]`` (first word is the length of the bytestring variable).
    
  • setup.py+6 1 modified
    @@ -64,7 +64,12 @@
         packages=find_packages(exclude=("tests", "docs")),
         python_requires=">=3.6",
         py_modules=["vyper"],
    -    install_requires=["asttokens==2.0.4", "pycryptodome>=3.5.1,<4", "semantic-version==2.8.5"],
    +    install_requires=[
    +        "asttokens==2.0.4",
    +        "pycryptodome>=3.5.1,<4",
    +        "semantic-version==2.8.5",
    +        "cached-property==1.5.2 ; python_version<'3.8'",
    +    ],
         setup_requires=["pytest-runner"],
         tests_require=extras_require["test"],
         extras_require=extras_require,
    
  • tests/cli/outputs/test_storage_layout.py+1 1 modified
    @@ -34,5 +34,5 @@ def public_bar():
                 "slot": 3,
             },
             "baz": {"type": "Bytes[65]", "location": "storage", "slot": 4},
    -        "bar": {"type": "uint256", "location": "storage", "slot": 9},
    +        "bar": {"type": "uint256", "location": "storage", "slot": 8},
         }
    
  • tests/compiler/test_source_map.py+1 1 modified
    @@ -32,7 +32,7 @@ def test_jump_map():
         pos_map = source_map["pc_pos_map"]
         jump_map = source_map["pc_jump_map"]
     
    -    assert len([v for v in jump_map.values() if v == "o"]) == 3
    +    assert len([v for v in jump_map.values() if v == "o"]) == 1
         assert len([v for v in jump_map.values() if v == "i"]) == 2
     
         code_lines = [i + "\n" for i in TEST_CODE.split("\n")]
    
  • tests/conftest.py+5 2 modified
    @@ -3,6 +3,7 @@
     
     import pytest
     from eth_tester import EthereumTester
    +from eth_utils import setup_DEBUG2_logging
     from hexbytes import HexBytes
     from web3 import Web3
     from web3.providers.eth_tester import EthereumTesterProvider
    @@ -26,12 +27,14 @@
     
     
     def set_evm_verbose_logging():
    -    logger = logging.getLogger("evm")
    -    logger.setLevel("TRACE")
    +    logger = logging.getLogger("eth.vm.computation.Computation")
    +    setup_DEBUG2_logging()
    +    logger.setLevel("DEBUG2")
     
     
     # Useful options to comment out whilst working:
     # set_evm_verbose_logging()
    +#
     # from vdb import vdb
     # vdb.set_evm_opcode_debugger()
     
    
  • tests/functional/codegen/test_abi_encode.py+1 1 modified
    @@ -31,7 +31,7 @@ def abi_encode(
         pet_metadata: bytes32,
         ensure_tuple: bool,
         include_method_id: bool
    -) -> Bytes[260]:
    +) -> Bytes[548]:
         human: Human = Human({
           name: name,
           pet: Animal({
    
  • tests/functional/codegen/test_struct_return.py+12 16 modified
    @@ -5,33 +5,29 @@ def test_nested_tuple(get_contract):
         code = """
     struct Animal:
         location: address
    -    fur: uint256
    +    fur: String[32]
     
     struct Human:
         location: address
    -    height: uint256
    +    animal: Animal
     
     @external
    -def return_nested_tuple() -> (Animal, Human):
    -    animal: Animal = Animal({
    -        location: 0x1234567890123456789012345678901234567890,
    -        fur: 123
    -    })
    -    human: Human = Human({
    -        location: 0x1234567890123456789012345678900000000000,
    -        height: 456
    -    })
    +def modify_nested_tuple(_human: Human) -> Human:
    +    human: Human = _human
     
         # do stuff, edit the structs
    -    animal.fur += 1
    -    human.height += 1
    +    # (13 is the length of the result)
    +    human.animal.fur = slice(concat(human.animal.fur, " is great"), 0, 13)
     
    -    return animal, human
    +    return human
         """
         c = get_contract(code)
         addr1 = "0x1234567890123456789012345678901234567890"
         addr2 = "0x1234567890123456789012345678900000000000"
    -    assert c.return_nested_tuple() == [(addr1, 124), (addr2, 457)]
    +    # assert c.modify_nested_tuple([addr1, 123], [addr2, 456]) == [[addr1, 124], [addr2, 457]]
    +    assert c.modify_nested_tuple(
    +        {"location": addr1, "animal": {"location": addr2, "fur": "wool"}}
    +    ) == (addr1, (addr2, "wool is great"),)
     
     
     @pytest.mark.parametrize("string", ["a", "abc", "abcde", "potato"])
    @@ -61,4 +57,4 @@ def test_values(a: address) -> Person:
         """
     
         c2 = get_contract(code)
    -    assert c2.test_values(c1.address) == [string, 42]
    +    assert c2.test_values(c1.address) == (string, 42)
    
  • tests/functional/context/types/test_size_in_bytes.py+1 2 modified
    @@ -24,8 +24,7 @@ def test_array_value_types(build_node, type_str, location, length, size):
         node = build_node(f"{type_str}[{length}]")
         type_definition = get_type_from_annotation(node, location)
     
    -    # TODO once storage of bytes is optimized, remove the +32
    -    assert type_definition.size_in_bytes == size + 32
    +    assert type_definition.size_in_bytes == size
     
     
     @pytest.mark.parametrize("type_str", BASE_TYPES)
    
  • tests/functional/test_storage_slots.py+8 8 modified
    @@ -55,18 +55,18 @@ def with_other_lock():
     
     def test_storage_slots(get_contract):
         c = get_contract(code)
    -    assert c.a() == ["ok", [4, 5, 6]]
    +    assert c.a() == ("ok", [4, 5, 6])
         assert [c.b(i) for i in range(2)] == [7, 8]
         assert c.c() == b"thisisthirtytwobytesokhowdoyoudo"
         assert [c.d(i) for i in range(4)] == [-1, -2, -3, -4]
         assert c.e() == "A realllllly long string but we wont use it all"
         assert c.f(0) == 33
    -    assert c.g(0) == [b"hello", [-66, 420], "another string"]
    -    assert c.g(1) == [
    +    assert c.g(0) == (b"hello", [-66, 420], "another string")
    +    assert c.g(1) == (
             b"gbye",
             [1337, 888],
             "whatifthisstringtakesuptheentirelengthwouldthatbesobadidothinkso",
    -    ]
    +    )
         assert [c.foo(0, i) for i in range(3)] == [987, 654, 321]
         assert [c.foo(1, i) for i in range(3)] == [123, 456, 789]
         assert c.h(0) == 123456789
    @@ -80,18 +80,18 @@ def test_reentrancy_lock(get_contract):
         c.with_lock()
         c.with_other_lock()
     
    -    assert c.a() == ["ok", [4, 5, 6]]
    +    assert c.a() == ("ok", [4, 5, 6])
         assert [c.b(i) for i in range(2)] == [7, 8]
         assert c.c() == b"thisisthirtytwobytesokhowdoyoudo"
         assert [c.d(i) for i in range(4)] == [-1, -2, -3, -4]
         assert c.e() == "A realllllly long string but we wont use it all"
         assert c.f(0) == 33
    -    assert c.g(0) == [b"hello", [-66, 420], "another string"]
    -    assert c.g(1) == [
    +    assert c.g(0) == (b"hello", [-66, 420], "another string")
    +    assert c.g(1) == (
             b"gbye",
             [1337, 888],
             "whatifthisstringtakesuptheentirelengthwouldthatbesobadidothinkso",
    -    ]
    +    )
         assert [c.foo(0, i) for i in range(3)] == [987, 654, 321]
         assert [c.foo(1, i) for i in range(3)] == [123, 456, 789]
         assert c.h(0) == 123456789
    
  • tests/parser/exceptions/test_argument_exception.py+0 25 modified
    @@ -89,31 +89,6 @@ def foo():
         for i in range(1, 2, 3, 4):
             pass
         """,
    -    """
    -struct Foo:
    -    a: Bytes[32]
    -
    -@external
    -def foo(a: Foo):
    -    pass
    -    """,
    -    """
    -struct Foo:
    -    a: String[32]
    -
    -@external
    -def foo(a: Foo):
    -    pass
    -    """,
    -    """
    -struct Foo:
    -    b: uint256
    -    a: String[32]
    -
    -@external
    -def foo(a: Foo):
    -    pass
    -    """,
     ]
     
     
    
  • tests/parser/features/decorators/test_private.py+1 1 modified
    @@ -580,7 +580,7 @@ def foo() -> A:
         return self._foo([1, 2, 3, 4], 5)
         """,
             (),
    -        [[1, 2, 3, 4], 5],
    +        ([1, 2, 3, 4], 5),
         ),
         (
             """
    
  • tests/parser/features/external_contracts/test_external_contract_calls.py+2 2 modified
    @@ -829,7 +829,7 @@ def test(addr: address) -> (int128, address):
         c1 = get_contract_with_gas_estimation(contract_1)
         c2 = get_contract_with_gas_estimation(contract_2)
     
    -    assert c1.out_literals() == [1, "0x0000000000000000000000000000000000012345"]
    +    assert c1.out_literals() == (1, "0x0000000000000000000000000000000000012345")
         assert c2.test(c1.address) == list(c1.out_literals())
     
     
    @@ -862,7 +862,7 @@ def test(addr: address) -> (int128, String[{ln}], Bytes[{ln}]):
         c1 = get_contract_with_gas_estimation(contract_1)
         c2 = get_contract_with_gas_estimation(contract_2)
     
    -    assert c1.get_struct_x() == [i, s, bytes(s, "utf-8")]
    +    assert c1.get_struct_x() == (i, s, bytes(s, "utf-8"))
         assert c2.test(c1.address) == list(c1.get_struct_x())
     
     
    
  • tests/parser/features/external_contracts/test_self_call_struct.py+4 4 modified
    @@ -24,11 +24,11 @@ def wrap_get_my_struct_BROKEN(_e1: decimal) -> MyStruct:
         return self.get_my_struct(_e1, block.timestamp)
         """
         c = get_contract(code)
    -    assert c.wrap_get_my_struct_WORKING(Decimal("0.1")) == [
    +    assert c.wrap_get_my_struct_WORKING(Decimal("0.1")) == (
             Decimal("0.1"),
             w3.eth.getBlock(w3.eth.blockNumber)["timestamp"],
    -    ]
    -    assert c.wrap_get_my_struct_BROKEN(Decimal("0.1")) == [
    +    )
    +    assert c.wrap_get_my_struct_BROKEN(Decimal("0.1")) == (
             Decimal("0.1"),
             w3.eth.getBlock(w3.eth.blockNumber)["timestamp"],
    -    ]
    +    )
    
  • tests/parser/features/iteration/test_repeater.py+4 2 modified
    @@ -106,8 +106,10 @@ def test_return_inside_repeater(get_contract, typ):
     @internal
     def _final(a: {typ}) -> {typ}:
         for i in range(10):
    -        if i > a:
    -            return i
    +        for j in range(10):
    +            if j > 5:
    +                if i > a:
    +                    return i
         return 31337
     
     @internal
    
  • tests/parser/features/test_init.py+3 2 modified
    @@ -14,9 +14,10 @@ def __init__(a: uint256):
     
         assert c.val() == 123
     
    -    # Make sure the init signature has no unecessary CALLDATLOAD copy.
    +    # Make sure the init code does not access calldata
         opcodes = vyper.compile_code(code, ["opcodes"])["opcodes"].split(" ")
         lll_return_idx = opcodes.index("JUMP")
     
    -    assert "CALLDATALOAD" in opcodes
    +    assert "CALLDATACOPY" in opcodes
    +    assert "CALLDATACOPY" not in opcodes[:lll_return_idx]
         assert "CALLDATALOAD" not in opcodes[:lll_return_idx]
    
  • tests/parser/functions/test_abi.py+14 6 modified
    @@ -75,13 +75,21 @@ def foo(s: MyStruct) -> MyStruct:
         func_abi = abi[0]
     
         assert func_abi["name"] == "foo"
    -    expected = {
    +
    +    expected_output = [
    +        {
    +            "type": "tuple",
    +            "name": "",
    +            "components": [{"type": "address", "name": "a"}, {"type": "uint256", "name": "b"}],
    +        }
    +    ]
    +
    +    assert func_abi["outputs"] == expected_output
    +
    +    expected_input = {
             "type": "tuple",
    -        "name": "",
    +        "name": "s",
             "components": [{"type": "address", "name": "a"}, {"type": "uint256", "name": "b"}],
         }
     
    -    assert func_abi["outputs"] == expected["components"]
    -
    -    expected["name"] = "s"
    -    assert func_abi["inputs"][0] == expected
    +    assert func_abi["inputs"][0] == expected_input
    
  • tests/parser/functions/test_return_struct.py+6 6 modified
    @@ -24,7 +24,7 @@ def test() -> Voter:
     
         c = get_contract_with_gas_estimation(code)
     
    -    assert c.test() == [123, True]
    +    assert c.test() == (123, True)
     
     
     def test_struct_return(get_contract_with_gas_estimation):
    @@ -74,13 +74,13 @@ def pub6() -> Foo:
         foo: Foo = Foo({x: 123, y: 456})
         return self.return_arg(foo)
         """
    -    foo = [123, 456]
    +    foo = (123, 456)
     
         c = get_contract_with_gas_estimation(code)
     
    -    assert c.pub1() == [1, 2]
    -    assert c.pub2() == [3, 4]
    -    assert c.pub3() == [5, 6]
    -    assert c.pub4() == [7, 8]
    +    assert c.pub1() == (1, 2)
    +    assert c.pub2() == (3, 4)
    +    assert c.pub3() == (5, 6)
    +    assert c.pub4() == (7, 8)
         assert c.pub5(foo) == foo
         assert c.pub6() == foo
    
  • tests/parser/types/numbers/test_constants.py+3 2 modified
    @@ -175,11 +175,12 @@ def test_constant_folds(search_for_sublist):
     def test() -> uint256:
         # calculate some constant which is really unlikely to be randomly
         # in bytecode
    -    return 2**SOME_CONSTANT * SOME_PRIME
    +    ret: uint256 = 2**SOME_CONSTANT * SOME_PRIME
    +    return ret
         """
     
         lll = compile_code(code, ["ir"])["ir"]
    -    assert search_for_sublist(lll, ["mstore", [0], [2 ** 12 * some_prime]])
    +    assert search_for_sublist(lll, ["mstore", [320], [2 ** 12 * some_prime]])
     
     
     def test_constant_lists(get_contract):
    
  • tests/parser/types/test_node_types.py+2 2 modified
    @@ -60,8 +60,8 @@ def test_canonicalize_type():
     
     def test_get_size_of_type():
         assert get_size_of_type(BaseType("int128")) == 1
    -    assert get_size_of_type(ByteArrayType(12)) == 3
    -    assert get_size_of_type(ByteArrayType(33)) == 4
    +    assert get_size_of_type(ByteArrayType(12)) == 2
    +    assert get_size_of_type(ByteArrayType(33)) == 3
         assert get_size_of_type(ListType(BaseType("int128"), 10)) == 10
     
         _tuple = TupleType([BaseType("int128"), BaseType("decimal")])
    
  • vyper/ast/signatures/function_signature.py+97 153 modified
    @@ -1,29 +1,29 @@
     import math
    -from collections import Counter
    +from dataclasses import dataclass
     
     from vyper import ast as vy_ast
    -from vyper.exceptions import FunctionDeclarationException, StructureException
    -from vyper.old_codegen.lll_node import LLLnode
    -from vyper.old_codegen.parser_utils import check_single_exit, getpos
    +from vyper.exceptions import StructureException
    +from vyper.old_codegen.lll_node import Encoding
     from vyper.old_codegen.types import (
    -    ByteArrayLike,
    -    TupleType,
    +    NodeType,
         canonicalize_type,
         get_size_of_type,
         parse_type,
     )
    -from vyper.utils import fourbytes_to_int, keccak256
    +from vyper.utils import cached_property, mkalphanum
     
     
    -# Function argument
    +# Function variable
    +# TODO move to context.py
    +# TODO use dataclass
     class VariableRecord:
         def __init__(
             self,
             name,
             pos,
             typ,
             mutable,
    -        *,
    +        encoding=Encoding.VYPER,
             location="memory",
             blockscopes=None,
             defined_at=None,
    @@ -34,10 +34,16 @@ def __init__(
             self.typ = typ
             self.mutable = mutable
             self.location = location
    +        self.encoding = encoding
             self.blockscopes = [] if blockscopes is None else blockscopes
             self.defined_at = defined_at  # source code location variable record was defined.
             self.is_internal = is_internal
     
    +    def __repr__(self):
    +        ret = vars(self)
    +        ret["allocated"] = self.size * 32
    +        return f"VariableRecord(f{ret})"
    +
         @property
         def size(self):
             if hasattr(self.typ, "size_in_bytes"):
    @@ -53,131 +59,122 @@ def __init__(self, *args):
             super(ContractRecord, self).__init__(*args)
     
     
    +@dataclass
    +class FunctionArg:
    +    name: str
    +    typ: NodeType
    +    ast_source: vy_ast.VyperNode
    +
    +
     # Function signature object
     class FunctionSignature:
         def __init__(
             self,
             name,
             args,
    -        output_type,
    +        return_type,
             mutability,
             internal,
             nonreentrant_key,
    -        sig,
    -        method_id,
             func_ast_code,
             is_from_json,
         ):
             self.name = name
             self.args = args
    -        self.output_type = output_type
    +        self.return_type = return_type
             self.mutability = mutability
             self.internal = internal
    -        self.sig = sig
    -        self.method_id = method_id
             self.gas = None
             self.nonreentrant_key = nonreentrant_key
             self.func_ast_code = func_ast_code
             self.is_from_json = is_from_json
    -        self.calculate_arg_totals()
    +
    +        self.set_default_args()
     
         def __str__(self):
             input_name = "def " + self.name + "(" + ",".join([str(arg.typ) for arg in self.args]) + ")"
    -        if self.output_type:
    -            return input_name + " -> " + str(self.output_type) + ":"
    +        if self.return_type:
    +            return input_name + " -> " + str(self.return_type) + ":"
             return input_name + ":"
     
    -    def calculate_arg_totals(self):
    -        """ Calculate base arguments, and totals. """
    -
    -        code = self.func_ast_code
    -        self.base_args = []
    -        self.total_default_args = 0
    -
    -        if hasattr(code.args, "defaults"):
    -            self.total_default_args = len(code.args.defaults)
    -            if self.total_default_args > 0:
    -                # all argument w/o defaults
    -                self.base_args = self.args[: -self.total_default_args]
    -            else:
    -                # No default args, so base_args = args.
    -                self.base_args = self.args
    -            # All default argument name/type definitions.
    -            self.default_args = code.args.args[-self.total_default_args :]  # noqa: E203
    -            # Keep all the value to assign to default parameters.
    -            self.default_values = dict(
    -                zip([arg.arg for arg in self.default_args], code.args.defaults)
    -            )
    -
    -        # Calculate the total sizes in memory the function arguments will take use.
    -        # Total memory size of all arguments (base + default together).
    -        self.max_copy_size = sum(
    -            [
    -                32 if isinstance(arg.typ, ByteArrayLike) else get_size_of_type(arg.typ) * 32
    -                for arg in self.args
    -            ]
    -        )
    -        # Total memory size of base arguments (arguments exclude default parameters).
    -        self.base_copy_size = sum(
    -            [
    -                32 if isinstance(arg.typ, ByteArrayLike) else get_size_of_type(arg.typ) * 32
    -                for arg in self.base_args
    -            ]
    -        )
    +    @cached_property
    +    def _lll_identifier(self) -> str:
    +        # we could do a bit better than this but it just needs to be unique
    +        visibility = "internal" if self.internal else "external"
    +        argz = ",".join([str(arg.typ) for arg in self.args])
    +        ret = f"{visibility} {self.name} ({argz})"
    +        return mkalphanum(ret)
    +
    +    # calculate the abi signature for a given set of kwargs
    +    def abi_signature_for_kwargs(self, kwargs):
    +        args = self.base_args + kwargs
    +        return self.name + "(" + ",".join([canonicalize_type(arg.typ) for arg in args]) + ")"
    +
    +    @cached_property
    +    def base_signature(self):
    +        return self.abi_signature_for_kwargs([])
    +
    +    @property
    +    # common entry point for external function with kwargs
    +    def external_function_base_entry_label(self):
    +        assert not self.internal
    +
    +        return self._lll_identifier + "_common"
    +
    +    @property
    +    def internal_function_label(self):
    +        assert self.internal, "why are you doing this"
     
    -    # Get the canonical function signature
    -    @staticmethod
    -    def get_full_sig(func_name, args, sigs, custom_structs):
    -        def get_type(arg):
    -            if isinstance(arg, LLLnode):
    -                return canonicalize_type(arg.typ)
    -            elif hasattr(arg, "annotation"):
    -                return canonicalize_type(
    -                    parse_type(arg.annotation, None, sigs, custom_structs=custom_structs,)
    -                )
    +        return self._lll_identifier
     
    -        return func_name + "(" + ",".join([get_type(arg) for arg in args]) + ")"
    +    @property
    +    def exit_sequence_label(self):
    +        return self._lll_identifier + "_cleanup"
    +
    +    def set_default_args(self):
    +        """Split base from kwargs and set member data structures"""
    +
    +        args = self.func_ast_code.args
    +
    +        defaults = getattr(args, "defaults", [])
    +        num_base_args = len(args.args) - len(defaults)
    +
    +        self.base_args = self.args[:num_base_args]
    +        self.default_args = self.args[num_base_args:]
    +
    +        # Keep all the value to assign to default parameters.
    +        self.default_values = dict(zip([arg.name for arg in self.default_args], defaults))
     
         # Get a signature from a function definition
         @classmethod
         def from_definition(
             cls,
    -        code,
    -        sigs=None,
    +        func_ast,  # vy_ast.FunctionDef
    +        sigs=None,  # TODO replace sigs and custom_structs with GlobalContext?
             custom_structs=None,
             interface_def=False,
    -        constant_override=False,
    +        constant_override=False,  # CMC 20210907 what does this do?
             is_from_json=False,
         ):
    -        if not custom_structs:
    +        if custom_structs is None:
                 custom_structs = {}
     
    -        name = code.name
    -        mem_pos = 0
    +        name = func_ast.name
     
    -        # Determine the arguments, expects something of the form def foo(arg1:
    -        # int128, arg2: int128 ...
             args = []
    -        for arg in code.args.args:
    -            # Each arg needs a type specified.
    -            typ = arg.annotation
    -            parsed_type = parse_type(typ, None, sigs, custom_structs=custom_structs,)
    -            args.append(
    -                VariableRecord(arg.arg, mem_pos, parsed_type, False, defined_at=getpos(arg),)
    -            )
    -
    -            if isinstance(parsed_type, ByteArrayLike):
    -                mem_pos += 32
    -            else:
    -                mem_pos += get_size_of_type(parsed_type) * 32
    +        for arg in func_ast.args.args:
    +            argname = arg.arg
    +            argtyp = parse_type(arg.annotation, None, sigs, custom_structs=custom_structs,)
    +
    +            args.append(FunctionArg(argname, argtyp, arg))
     
             mutability = "nonpayable"  # Assume nonpayable by default
    -        nonreentrant_key = ""
    -        is_internal = False
    +        nonreentrant_key = None
    +        is_internal = None
     
             # Update function properties from decorators
             # NOTE: Can't import enums here because of circular import
    -        for dec in code.decorator_list:
    +        for dec in func_ast.decorator_list:
                 if isinstance(dec, vy_ast.Name) and dec.id in ("payable", "view", "pure"):
                     mutability = dec.id
                 elif isinstance(dec, vy_ast.Name) and dec.id == "internal":
    @@ -199,80 +196,27 @@ def from_definition(
             # def foo() -> int128: ...
             # If there is no return type, ie. it's of the form def foo(): ...
             # and NOT def foo() -> type: ..., then it's null
    -        output_type = None
    -        if code.returns:
    -            output_type = parse_type(code.returns, None, sigs, custom_structs=custom_structs,)
    -            # Output type must be canonicalizable
    -            assert isinstance(output_type, TupleType) or canonicalize_type(output_type)
    -        # Get the canonical function signature
    -        sig = cls.get_full_sig(name, code.args.args, sigs, custom_structs)
    -
    -        # Take the first 4 bytes of the hash of the sig to get the method ID
    -        method_id = fourbytes_to_int(keccak256(bytes(sig, "utf-8"))[:4])
    +        return_type = None
    +        if func_ast.returns:
    +            return_type = parse_type(func_ast.returns, None, sigs, custom_structs=custom_structs,)
    +            # sanity check: Output type must be canonicalizable
    +            assert canonicalize_type(return_type)
    +
             return cls(
                 name,
                 args,
    -            output_type,
    +            return_type,
                 mutability,
                 is_internal,
                 nonreentrant_key,
    -            sig,
    -            method_id,
    -            code,
    +            func_ast,
                 is_from_json,
             )
     
    -    @classmethod
    -    def lookup_sig(cls, sigs, method_name, expr_args, stmt_or_expr, context):
    -        """
    -        Using a list of args, determine the most accurate signature to use from
    -        the given context
    -        """
    -
    -        def synonymise(s):
    -            return s.replace("int128", "num").replace("uint256", "num")
    -
    -        # for sig in sigs['self']
    -        full_sig = cls.get_full_sig(stmt_or_expr.func.attr, expr_args, None, context.structs,)
    -        method_names_dict = dict(Counter([x.split("(")[0] for x in context.sigs["self"]]))
    -        if method_name not in method_names_dict:
    -            raise FunctionDeclarationException(
    -                "Function not declared yet (reminder: functions cannot "
    -                f"call functions later in code than themselves): {method_name}"
    -            )
    -
    -        if method_names_dict[method_name] == 1:
    -            return next(
    -                sig
    -                for name, sig in context.sigs["self"].items()
    -                if name.split("(")[0] == method_name
    -            )
    -        if full_sig in context.sigs["self"]:
    -            return context.sigs["self"][full_sig]
    -        else:
    -            synonym_sig = synonymise(full_sig)
    -            syn_sigs_test = [synonymise(k) for k in context.sigs.keys()]
    -            if len(syn_sigs_test) != len(set(syn_sigs_test)):
    -                raise Exception(
    -                    "Incompatible default parameter signature,"
    -                    "can not tell the number type of literal",
    -                    stmt_or_expr,
    -                )
    -            synonym_sigs = [(synonymise(k), v) for k, v in context.sigs["self"].items()]
    -            ssig = [s[1] for s in synonym_sigs if s[0] == synonym_sig]
    -            if len(ssig) == 0:
    -                raise FunctionDeclarationException(
    -                    "Function not declared yet (reminder: functions cannot "
    -                    f"call functions later in code than themselves): {method_name}"
    -                )
    -            return ssig[0]
    -
    +    @property
         def is_default_func(self):
             return self.name == "__default__"
     
    -    def is_initializer(self):
    +    @property
    +    def is_init_func(self):
             return self.name == "__init__"
    -
    -    def validate_return_statement_balance(self):
    -        # Run balanced return statement check.
    -        check_single_exit(self.func_ast_code)
    
  • vyper/ast/signatures/interface.py+14 3 modified
    @@ -5,7 +5,6 @@
     
     import vyper.builtin_interfaces
     from vyper import ast as vy_ast
    -from vyper.ast.signatures import sig_utils
     from vyper.ast.signatures.function_signature import FunctionSignature
     from vyper.exceptions import StructureException
     from vyper.old_codegen.global_context import GlobalContext
    @@ -81,7 +80,7 @@ def mk_full_signature_from_json(abi):
                 decorator_list.append(vy_ast.Name(id="payable"))
     
             sig = FunctionSignature.from_definition(
    -            code=vy_ast.FunctionDef(
    +            func_ast=vy_ast.FunctionDef(
                     name=func["name"],
                     args=vy_ast.arguments(args=args),
                     decorator_list=decorator_list,
    @@ -94,6 +93,18 @@ def mk_full_signature_from_json(abi):
         return sigs
     
     
    +def _get_external_signatures(global_ctx, sig_formatter=lambda x: x):
    +    ret = []
    +
    +    for code in global_ctx._defs:
    +        sig = FunctionSignature.from_definition(
    +            code, sigs=global_ctx._contracts, custom_structs=global_ctx._structs,
    +        )
    +        if not sig.internal:
    +            ret.append(sig_formatter(sig))
    +    return ret
    +
    +
     def extract_sigs(sig_code, interface_name=None):
         if sig_code["type"] == "vyper":
             interface_ast = [
    @@ -116,7 +127,7 @@ def extract_sigs(sig_code, interface_name=None):
                 or (isinstance(i, vy_ast.AnnAssign) and i.target.id != "implements")
             ]
             global_ctx = GlobalContext.get_global_context(interface_ast)
    -        return sig_utils.mk_full_signature(global_ctx, sig_formatter=lambda x: x)
    +        return _get_external_signatures(global_ctx)
         elif sig_code["type"] == "json":
             return mk_full_signature_from_json(sig_code["code"])
         else:
    
  • vyper/ast/signatures/sig_utils.py+0 58 removed
    @@ -1,58 +0,0 @@
    -import copy
    -
    -from vyper.ast.signatures.function_signature import FunctionSignature
    -
    -
    -# Generate default argument function signatures.
    -def generate_default_arg_sigs(code, interfaces, global_ctx):
    -    # generate all sigs, and attach.
    -    total_default_args = len(code.args.defaults)
    -    if total_default_args == 0:
    -        return [
    -            FunctionSignature.from_definition(
    -                code, sigs=interfaces, custom_structs=global_ctx._structs,
    -            )
    -        ]
    -    base_args = code.args.args[:-total_default_args]
    -    default_args = code.args.args[-total_default_args:]
    -
    -    # Generate a list of default function combinations.
    -    row = [False] * (total_default_args)
    -    table = [row.copy()]
    -    for i in range(total_default_args):
    -        row[i] = True
    -        table.append(row.copy())
    -
    -    default_sig_strs = []
    -    sig_fun_defs = []
    -    for truth_row in table:
    -        new_code = copy.deepcopy(code)
    -        new_code.args.args = copy.deepcopy(base_args)
    -        new_code.args.default = []
    -        # Add necessary default args.
    -        for idx, val in enumerate(truth_row):
    -            if val is True:
    -                new_code.args.args.append(default_args[idx])
    -        sig = FunctionSignature.from_definition(
    -            new_code, sigs=interfaces, custom_structs=global_ctx._structs,
    -        )
    -        default_sig_strs.append(sig.sig)
    -        sig_fun_defs.append(sig)
    -
    -    return sig_fun_defs
    -
    -
    -# Get ABI signature
    -def mk_full_signature(global_ctx, sig_formatter):
    -    o = []
    -
    -    # Produce function signatures.
    -    for code in global_ctx._defs:
    -        sig = FunctionSignature.from_definition(
    -            code, sigs=global_ctx._contracts, custom_structs=global_ctx._structs,
    -        )
    -        if not sig.internal:
    -            default_sigs = generate_default_arg_sigs(code, global_ctx._contracts, global_ctx)
    -            for s in default_sigs:
    -                o.append(sig_formatter(s))
    -    return o
    
  • vyper/builtin_functions/convert.py+10 4 modified
    @@ -7,7 +7,12 @@
     from vyper.evm.opcodes import version_check
     from vyper.exceptions import InvalidLiteral, StructureException, TypeMismatch
     from vyper.old_codegen.arg_clamps import address_clamp, int128_clamp
    -from vyper.old_codegen.parser_utils import LLLnode, byte_array_to_num, getpos
    +from vyper.old_codegen.parser_utils import (
    +    LLLnode,
    +    byte_array_to_num,
    +    getpos,
    +    load_op,
    +)
     from vyper.old_codegen.types import (
         BaseType,
         ByteArrayType,
    @@ -325,10 +330,11 @@ def to_bytes32(expr, args, kwargs, context):
                     f"Unable to convert bytes[{_len}] to bytes32, max length is too " "large."
                 )
     
    -        if in_arg.location == "memory":
    -            return LLLnode.from_list(["mload", ["add", in_arg, 32]], typ=BaseType("bytes32"))
    -        elif in_arg.location == "storage":
    +        if in_arg.location == "storage":
                 return LLLnode.from_list(["sload", ["add", in_arg, 1]], typ=BaseType("bytes32"))
    +        else:
    +            op = load_op(in_arg.location)
    +            return LLLnode.from_list([op, ["add", in_arg, 32]], typ=BaseType("bytes32"))
     
         else:
             return LLLnode(
    
  • vyper/builtin_functions/functions.py+71 54 modified
    @@ -26,16 +26,17 @@
         abi_encode,
         abi_type_of,
         abi_type_of2,
    -    lll_tuple_from_args,
     )
     from vyper.old_codegen.arg_clamps import int128_clamp
     from vyper.old_codegen.expr import Expr
     from vyper.old_codegen.keccak256_helper import keccak256_helper
     from vyper.old_codegen.parser_utils import (
         LLLnode,
    -    add_variable_offset,
         get_bytearray_length,
    +    get_element_ptr,
         getpos,
    +    lll_tuple_from_args,
    +    load_op,
         make_byte_array_copier,
         make_byte_slice_copier,
         unwrap_location,
    @@ -254,6 +255,7 @@ def fetch_call_return(self, node):
         def build_LLL(self, expr, args, kwargs, context):
     
             sub, start, length = args
    +
             if is_base_type(sub.typ, "bytes32"):
                 if (start.typ.is_literal and length.typ.is_literal) and not (
                     0 <= start.value + length.value <= 32
    @@ -262,12 +264,6 @@ def build_LLL(self, expr, args, kwargs, context):
                         "Invalid start / length values needs to be between 0 and 32.", expr,
                     )
                 sub_typ_maxlen = 32
    -        elif sub.location == "calldata":
    -            # if we are slicing msg.data, the length should
    -            # be a constant, since msg.data can be of dynamic length
    -            # we can't use it's length as the maxlen
    -            assert isinstance(length.value, int)  # sanity check
    -            sub_typ_maxlen = length.value
             else:
                 sub_typ_maxlen = sub.typ.maxlen
     
    @@ -278,15 +274,19 @@ def build_LLL(self, expr, args, kwargs, context):
                 ReturnType = OldStringType
     
             # Node representing the position of the output in memory
    +        # CMC 20210917 shouldn't this be a variable with newmaxlen?
             np = context.new_internal_variable(ReturnType(maxlen=sub_typ_maxlen + 32))
    +        # TODO deallocate np
             placeholder_node = LLLnode.from_list(np, typ=sub.typ, location="memory")
             placeholder_plus_32_node = LLLnode.from_list(np + 32, typ=sub.typ, location="memory")
    -        # Copies over bytearray data
    -        if sub.location == "storage":
    -            adj_sub = LLLnode.from_list(
    -                ["add", sub, ["add", ["div", "_start", 32], 1]], typ=sub.typ, location=sub.location,
    -            )
    -        elif sub.location == "calldata":
    +
    +        # special handling for slice(msg.data)
    +        if sub.location == "calldata" and sub.value == 0:
    +            assert expr.args[0].value.id == "msg" and expr.args[0].attr == "data"
    +            # if we are slicing msg.data, the length should
    +            # be a constant, since msg.data can be of dynamic length
    +            # we can't use its length as the maxlen
    +            assert isinstance(length.value, int)  # sanity check
                 node = [
                     "seq",
                     ["assert", ["le", ["add", start, length], "calldatasize"]],  # runtime bounds check
    @@ -295,6 +295,14 @@ def build_LLL(self, expr, args, kwargs, context):
                     np,
                 ]
                 return LLLnode.from_list(node, typ=ByteArrayType(length.value), location="memory")
    +
    +        # Copy over bytearray data
    +        # CMC 20210917 how does this routine work?
    +
    +        if sub.location == "storage":
    +            adj_sub = LLLnode.from_list(
    +                ["add", sub, ["add", ["div", "_start", 32], 1]], typ=sub.typ, location=sub.location,
    +            )
             else:
                 adj_sub = LLLnode.from_list(
                     ["add", sub, ["add", ["sub", "_start", ["mod", "_start", 32]], 32]],
    @@ -308,16 +316,17 @@ def build_LLL(self, expr, args, kwargs, context):
             copier = make_byte_slice_copier(
                 placeholder_plus_32_node,
                 adj_sub,
    -            ["add", "_length", 32],
    +            ["add", "_length", 32],  # CMC 20210917 shouldn't this just be _length
                 sub_typ_maxlen,
                 pos=getpos(expr),
             )
    +
             # New maximum length in the type of the result
             newmaxlen = length.value if not len(length.args) else sub_typ_maxlen
             if is_base_type(sub.typ, "bytes32"):
                 maxlen = 32
             else:
    -            maxlen = ["mload", Expr(sub, context=context).lll_node]  # Retrieve length of the bytes.
    +            maxlen = get_bytearray_length(sub)
     
             out = [
                 "with",
    @@ -465,8 +474,10 @@ def build_LLL(self, expr, context):
                     if arg.typ.maxlen == 0:
                         continue
                     # Get the length of the current argument
    -                if arg.location == "memory":
    -                    length = LLLnode.from_list(["mload", "_arg"], typ=BaseType("int128"))
    +                if arg.location in ("memory", "calldata", "code"):
    +                    length = LLLnode.from_list(
    +                        [load_op(arg.location), "_arg"], typ=BaseType("int128")
    +                    )
                         argstart = LLLnode.from_list(
                             ["add", "_arg", 32], typ=arg.typ, location=arg.location,
                         )
    @@ -602,24 +613,18 @@ def build_LLL(self, expr, args, kwargs, context):
                     add_gas_estimate=SHA256_BASE_GAS + 1 * SHA256_PER_WORD_GAS,
                 )
             # bytearay-like input
    -        if sub.location == "storage":
    -            # Copy storage to memory
    -            placeholder = context.new_internal_variable(sub.typ)
    -            placeholder_node = LLLnode.from_list(placeholder, typ=sub.typ, location="memory")
    -            copier = make_byte_array_copier(
    -                placeholder_node, LLLnode.from_list("_sub", typ=sub.typ, location=sub.location),
    -            )
    +        # special case if it's already in memory
    +        if sub.location == "memory":
                 return LLLnode.from_list(
                     [
                         "with",
                         "_sub",
                         sub,
                         [
                             "seq",
    -                        copier,
                             _make_sha256_call(
    -                            inp_start=["add", placeholder, 32],
    -                            inp_len=["mload", placeholder],
    +                            inp_start=["add", "_sub", 32],
    +                            inp_len=["mload", "_sub"],
                                 out_start=MemoryPositions.FREE_VAR_SPACE,
                                 out_len=32,
                             ),
    @@ -630,17 +635,24 @@ def build_LLL(self, expr, args, kwargs, context):
                     pos=getpos(expr),
                     add_gas_estimate=SHA256_BASE_GAS + sub.typ.maxlen * SHA256_PER_WORD_GAS,
                 )
    -        elif sub.location == "memory":
    +        else:
    +            # otherwise, copy it to memory and then call the precompile
    +            placeholder = context.new_internal_variable(sub.typ)
    +            placeholder_node = LLLnode.from_list(placeholder, typ=sub.typ, location="memory")
    +            copier = make_byte_array_copier(
    +                placeholder_node, LLLnode.from_list("_sub", typ=sub.typ, location=sub.location),
    +            )
                 return LLLnode.from_list(
                     [
                         "with",
                         "_sub",
                         sub,
                         [
                             "seq",
    +                        copier,
                             _make_sha256_call(
    -                            inp_start=["add", "_sub", 32],
    -                            inp_len=["mload", "_sub"],
    +                            inp_start=["add", placeholder, 32],
    +                            inp_len=["mload", placeholder],
                                 out_start=MemoryPositions.FREE_VAR_SPACE,
                                 out_len=32,
                             ),
    @@ -651,9 +663,6 @@ def build_LLL(self, expr, args, kwargs, context):
                     pos=getpos(expr),
                     add_gas_estimate=SHA256_BASE_GAS + sub.typ.maxlen * SHA256_PER_WORD_GAS,
                 )
    -        else:
    -            # This should never happen, but just left here for future compiler-writers.
    -            raise Exception(f"Unsupported location: {sub.location}")  # pragma: no test
     
     
     class MethodID:
    @@ -742,8 +751,8 @@ def build_LLL(self, expr, args, kwargs, context):
             )
     
     
    -def avo(arg, ind, pos):
    -    return unwrap_location(add_variable_offset(arg, LLLnode.from_list(ind, "int128"), pos=pos))
    +def _getelem(arg, ind, pos):
    +    return unwrap_location(get_element_ptr(arg, LLLnode.from_list(ind, "int128"), pos=pos))
     
     
     class ECAdd(_SimpleBuiltinFunction):
    @@ -766,10 +775,10 @@ def build_LLL(self, expr, args, kwargs, context):
             o = LLLnode.from_list(
                 [
                     "seq",
    -                ["mstore", placeholder_node, avo(args[0], 0, pos)],
    -                ["mstore", ["add", placeholder_node, 32], avo(args[0], 1, pos)],
    -                ["mstore", ["add", placeholder_node, 64], avo(args[1], 0, pos)],
    -                ["mstore", ["add", placeholder_node, 96], avo(args[1], 1, pos)],
    +                ["mstore", placeholder_node, _getelem(args[0], 0, pos)],
    +                ["mstore", ["add", placeholder_node, 32], _getelem(args[0], 1, pos)],
    +                ["mstore", ["add", placeholder_node, 64], _getelem(args[1], 0, pos)],
    +                ["mstore", ["add", placeholder_node, 96], _getelem(args[1], 1, pos)],
                     ["assert", ["staticcall", ["gas"], 6, placeholder_node, 128, placeholder_node, 64]],
                     placeholder_node,
                 ],
    @@ -797,8 +806,8 @@ def build_LLL(self, expr, args, kwargs, context):
             o = LLLnode.from_list(
                 [
                     "seq",
    -                ["mstore", placeholder_node, avo(args[0], 0, pos)],
    -                ["mstore", ["add", placeholder_node, 32], avo(args[0], 1, pos)],
    +                ["mstore", placeholder_node, _getelem(args[0], 0, pos)],
    +                ["mstore", ["add", placeholder_node, 32], _getelem(args[0], 1, pos)],
                     ["mstore", ["add", placeholder_node, 64], args[1]],
                     ["assert", ["staticcall", ["gas"], 7, placeholder_node, 96, placeholder_node, 64]],
                     placeholder_node,
    @@ -810,10 +819,13 @@ def build_LLL(self, expr, args, kwargs, context):
             return o
     
     
    -def _memory_element_getter(index):
    -    return LLLnode.from_list(
    -        ["mload", ["add", "_sub", ["add", 32, ["mul", 32, index]]]], typ=BaseType("int128"),
    -    )
    +def _generic_element_getter(op):
    +    def f(index):
    +        return LLLnode.from_list(
    +            [op, ["add", "_sub", ["add", 32, ["mul", 32, index]]]], typ=BaseType("int128"),
    +        )
    +
    +    return f
     
     
     def _storage_element_getter(index):
    @@ -845,14 +857,14 @@ def build_LLL(self, expr, args, kwargs, context):
             sub, index = args
             ret_type = kwargs["output_type"]
             # Get length and specific element
    -        if sub.location == "memory":
    -            lengetter = LLLnode.from_list(["mload", "_sub"], typ=BaseType("int128"))
    -            elementgetter = _memory_element_getter
    -        elif sub.location == "storage":
    +        if sub.location == "storage":
                 lengetter = LLLnode.from_list(["sload", "_sub"], typ=BaseType("int128"))
                 elementgetter = _storage_element_getter
    -        # TODO: unclosed if/elif clause.  Undefined behavior if `sub.location`
    -        # isn't one of `memory`/`storage`
    +
    +        else:
    +            op = load_op(sub.location)
    +            lengetter = LLLnode.from_list([op, "_sub"], typ=BaseType("int128"))
    +            elementgetter = _generic_element_getter(op)
     
             # Special case: index known to be a multiple of 32
             if isinstance(index.value, int) and not index.value % 32:
    @@ -1492,7 +1504,7 @@ def build_LLL(self, expr, context):
     
     
     def get_create_forwarder_to_bytecode():
    -    # FLAG cyclic import?
    +    # NOTE cyclic import?
         from vyper.lll.compile_lll import assembly_to_evm
     
         loader_asm = [
    @@ -1757,7 +1769,8 @@ class ABIEncode(_SimpleBuiltinFunction):
         # to handle varargs.)
         # explanation of ensure_tuple:
         # default is to force even a single value into a tuple,
    -    # e.g. _abi_encode(bytes) -> abi_encode((bytes,))
    +    # e.g. _abi_encode(bytes) -> _abi_encode((bytes,))
    +    #      _abi_encode((bytes,)) -> _abi_encode(((bytes,),))
         # this follows the encoding convention for functions:
         # ://docs.soliditylang.org/en/v0.8.6/abi-spec.html#function-selector-and-argument-encoding
         # if this is turned off, then bytes will be encoded as bytes.
    @@ -1828,6 +1841,7 @@ def fetch_call_return(self, node):
             maxlen = arg_abi_t.size_bound()
     
             if self._method_id(node) is not None:
    +            # the output includes 4 bytes for the method_id.
                 maxlen += 4
     
             ret = BytesArrayDefinition()
    @@ -1851,6 +1865,8 @@ def build_LLL(self, expr, context):
     
             input_abi_t = abi_type_of(encode_input.typ)
             maxlen = input_abi_t.size_bound()
    +        if method_id is not None:
    +            maxlen += 4
     
             buf_t = ByteArrayType(maxlen=maxlen)
             buf = context.new_internal_variable(buf_t)
    @@ -1859,6 +1875,7 @@ def build_LLL(self, expr, context):
     
             ret = ["seq"]
             if method_id is not None:
    +            # <32 bytes length> | <4 bytes method_id> | <everything else>
                 # write the unaligned method_id first, then we will
                 # overwrite the 28 bytes of zeros with the bytestring length
                 ret += [["mstore", buf + 4, method_id]]
    
  • vyper/evm/opcodes.py+0 1 modified
    @@ -209,7 +209,6 @@
         "NE": (None, 2, 1, 6),
         "DEBUGGER": (None, 0, 0, 0),
         "LABEL": (None, 1, 0, 1),
    -    "GOTO": (None, 1, 0, 8),
     }
     
     COMB_OPCODES: OpcodeMap = {**OPCODES, **PSEUDO_OPCODES}
    
  • vyper/lll/compile_lll.py+56 10 modified
    @@ -493,9 +493,13 @@ def _compile_to_assembly(code, withargs=None, existing_labels=None, break_dest=N
                 break_dest,
                 height,
             )
    -    # # jump to a symbol
    +    # # jump to a symbol, and push variable arguments onto stack
         elif code.value == "goto":
    -        return ["_sym_" + str(code.args[0]), "JUMP"]
    +        o = []
    +        for i, c in enumerate(reversed(code.args[1:])):
    +            o.extend(_compile_to_assembly(c, withargs, existing_labels, break_dest, height + i))
    +        o.extend(["_sym_" + str(code.args[0]), "JUMP"])
    +        return o
         elif isinstance(code.value, str) and is_symbol(code.value):
             return [code.value]
         # set a symbol as a location.
    @@ -557,6 +561,22 @@ def _prune_unreachable_code(assembly):
                 i += 1
     
     
    +def _prune_inefficient_jumps(assembly):
    +    # prune sequences `_sym_x JUMP _sym_x JUMPDEST` to `_sym_x JUMPDEST`
    +    i = 0
    +    while i < len(assembly) - 4:
    +        if (
    +            is_symbol(assembly[i])
    +            and assembly[i + 1] == "JUMP"
    +            and assembly[i] == assembly[i + 2]
    +            and assembly[i + 3] == "JUMPDEST"
    +        ):
    +            # delete _sym_x JUMP
    +            del assembly[i : i + 2]  # noqa: E203
    +        else:
    +            i += 1
    +
    +
     def _merge_jumpdests(assembly):
         # When a nested subroutine finishes and is the final action within it's
         # parent subroutine, we end up with multiple simultaneous JUMPDEST
    @@ -593,13 +613,35 @@ def _merge_iszero(assembly):
                 i += 1
     
     
    -# Assembles assembly into EVM
    -def assembly_to_evm(assembly, start_pos=0):
    -    _prune_unreachable_code(assembly)
    +def _prune_unused_jumpdests(assembly):
    +    used_jumpdests = set()
     
    -    _merge_iszero(assembly)
    +    # find all used jumpdests
    +    for i in range(len(assembly) - 1):
    +        if is_symbol(assembly[i]) and assembly[i + 1] != "JUMPDEST":
    +            used_jumpdests.add(assembly[i])
    +
    +    # delete jumpdests that aren't used
    +    i = 0
    +    while i < len(assembly) - 2:
    +        if is_symbol(assembly[i]) and assembly[i] not in used_jumpdests:
    +            del assembly[i : i + 2]  # noqa: E203
    +        else:
    +            i += 1
     
    +
    +# optimize assembly, in place
    +def _optimize_assembly(assembly):
    +    _prune_unreachable_code(assembly)
    +    _merge_iszero(assembly)
         _merge_jumpdests(assembly)
    +    _prune_inefficient_jumps(assembly)
    +    _prune_unused_jumpdests(assembly)
    +
    +
    +# Assembles assembly into EVM
    +def assembly_to_evm(assembly, start_pos=0):
    +    _optimize_assembly(assembly)
     
         line_number_map = {
             "breakpoints": set(),
    @@ -622,11 +664,15 @@ def assembly_to_evm(assembly, start_pos=0):
     
             if item == "JUMP":
                 last = assembly[i - 1]
    -            if last == "MLOAD":
    -                line_number_map["pc_jump_map"][pos] = "o"
    -            elif is_symbol(last) and "_priv_" in last:
    -                line_number_map["pc_jump_map"][pos] = "i"
    +            if is_symbol(last) and last.startswith("_sym_internal"):
    +                if last.endswith("cleanup"):
    +                    # exit an internal function
    +                    line_number_map["pc_jump_map"][pos] = "o"
    +                else:
    +                    # enter an internal function
    +                    line_number_map["pc_jump_map"][pos] = "i"
                 else:
    +                # everything else
                     line_number_map["pc_jump_map"][pos] = "-"
             elif item in ("JUMPI", "JUMPDEST"):
                 line_number_map["pc_jump_map"][pos] = "-"
    
  • vyper/lll/optimizer.py+1 18 modified
    @@ -1,5 +1,5 @@
     import operator
    -from typing import Any, List, Optional
    +from typing import List, Optional
     
     from vyper.old_codegen.parser_utils import LLLnode
     from vyper.utils import LOADED_LIMITS
    @@ -191,23 +191,6 @@ def apply_general_optimizations(node: LLLnode) -> LLLnode:
                 annotation=node.annotation,
                 # let from_list handle valency and gas_estimate
             )
    -    elif node.value == "seq":
    -        xs: List[Any] = []
    -        for arg in argz:
    -            if arg.value == "seq":
    -                xs.extend(arg.args)
    -            else:
    -                xs.append(arg)
    -        return LLLnode(
    -            node.value,
    -            xs,
    -            node.typ,
    -            node.location,
    -            node.pos,
    -            node.annotation,
    -            add_gas_estimate=node.add_gas_estimate,
    -            valency=node.valency,
    -        )
         elif node.total_gas is not None:
             o = LLLnode(
                 node.value,
    
  • vyper/old_codegen/abi.py+106 87 modified
    @@ -1,8 +1,10 @@
     import vyper.semantics.types as vy
     from vyper.exceptions import CompilerPanic
    -from vyper.old_codegen.lll_node import LLLnode
    +from vyper.old_codegen.lll_node import Encoding, LLLnode
     from vyper.old_codegen.parser_utils import (
    -    add_variable_offset,
    +    _needs_clamp,
    +    clamp_basetype,
    +    get_element_ptr,
         make_setter,
         unwrap_location,
         zero_pad,
    @@ -14,7 +16,6 @@
         ListType,
         StringType,
         TupleLike,
    -    TupleType,
     )
     from vyper.utils import ceil32
     
    @@ -26,30 +27,55 @@ def is_dynamic(self):
             raise NotImplementedError("ABIType.is_dynamic")
     
         # size (in bytes) in the static section (aka 'head')
    -    # when embedded in a tuple.
    +    # when embedded in a complex type.
         def embedded_static_size(self):
             return 32 if self.is_dynamic() else self.static_size()
     
    +    # size bound in the dynamic section (aka 'tail')
    +    # when embedded in a complex type.
    +    def embedded_dynamic_size_bound(self):
    +        if not self.is_dynamic():
    +            return 0
    +        return self.size_bound()
    +
    +    def embedded_min_dynamic_size(self):
    +        if not self.is_dynamic():
    +            return 0
    +        return self.min_size()
    +
         # size (in bytes) of the static section
         def static_size(self):
             raise NotImplementedError("ABIType.static_size")
     
         # max size (in bytes) in the dynamic section (aka 'tail')
         def dynamic_size_bound(self):
    -        return 0
    +        if not self.is_dynamic():
    +            return 0
    +        raise NotImplementedError("ABIType.dynamic_size_bound")
     
         def size_bound(self):
             return self.static_size() + self.dynamic_size_bound()
     
    +    def min_size(self):
    +        return self.static_size() + self.min_dynamic_size()
    +
    +    def min_dynamic_size(self):
    +        if not self.is_dynamic():
    +            return 0
    +        raise NotImplementedError("ABIType.min_dynamic_size")
    +
         # The canonical name of the type for calculating the function selector
         def selector_name(self):
             raise NotImplementedError("ABIType.selector_name")
     
         # Whether the type is a tuple at the ABI level.
         # (This is important because if it does, it needs an offset.
         #   Compare the difference in encoding between `bytes` and `(bytes,)`.)
    -    def is_tuple(self):
    -        raise NotImplementedError("ABIType.is_tuple")
    +    def is_complex_type(self):
    +        raise NotImplementedError("ABIType.is_complex_type")
    +
    +    def __repr__(self):
    +        return str({type(self).__name__: vars(self)})
     
     
     # uint<M>: unsigned integer type of M bits, 0 < M <= 256, M % 8 == 0. e.g. uint32, uint8, uint256.
    @@ -71,7 +97,7 @@ def static_size(self):
         def selector_name(self):
             return ("" if self.signed else "u") + f"int{self.m_bits}"
     
    -    def is_tuple(self):
    +    def is_complex_type(self):
             return False
     
     
    @@ -122,7 +148,7 @@ def static_size(self):
         def selector_name(self):
             return ("" if self.signed else "u") + "fixed{self.m_bits}x{self.n_places}"
     
    -    def is_tuple(self):
    +    def is_complex_type(self):
             return False
     
     
    @@ -143,7 +169,7 @@ def static_size(self):
         def selector_name(self):
             return f"bytes{self.m_bytes}"
     
    -    def is_tuple(self):
    +    def is_complex_type(self):
             return False
     
     
    @@ -173,12 +199,15 @@ def static_size(self):
             return self.m_elems * self.subtyp.static_size()
     
         def dynamic_size_bound(self):
    -        return self.m_elems * self.subtyp.dynamic_size_bound()
    +        return self.m_elems * self.subtyp.embedded_dynamic_size_bound()
    +
    +    def min_dynamic_size(self):
    +        return self.m_elems * self.subtyp.embedded_min_dynamic_size()
     
         def selector_name(self):
             return f"{self.subtyp.selector_name()}[{self.m_elems}]"
     
    -    def is_tuple(self):
    +    def is_complex_type(self):
             return True
     
     
    @@ -201,10 +230,13 @@ def dynamic_size_bound(self):
             # length word + data
             return 32 + ceil32(self.bytes_bound)
     
    +    def min_dynamic_size(self):
    +        return 32
    +
         def selector_name(self):
             return "bytes"
     
    -    def is_tuple(self):
    +    def is_complex_type(self):
             return False
     
     
    @@ -228,12 +260,17 @@ def static_size(self):
             return 32
     
         def dynamic_size_bound(self):
    -        return self.subtyp.dynamic_size_bound() * self.elems_bound
    +        # TODO double check me
    +        return self.subtyp.embedded_dynamic_size_bound() * self.elems_bound
    +
    +    def min_dynamic_size(self):
    +        # TODO double check me
    +        return 32
     
         def selector_name(self):
             return f"{self.subtyp.selector_name()}[]"
     
    -    def is_tuple(self):
    +    def is_complex_type(self):
             return False
     
     
    @@ -248,9 +285,12 @@ def static_size(self):
             return sum([t.embedded_static_size() for t in self.subtyps])
     
         def dynamic_size_bound(self):
    -        return sum([t.dynamic_size_bound() for t in self.subtyps])
    +        return sum([t.embedded_dynamic_size_bound() for t in self.subtyps])
    +
    +    def min_dynamic_size(self):
    +        return sum([t.embedded_min_dynamic_size() for t in self.subtyps])
     
    -    def is_tuple(self):
    +    def is_complex_type(self):
             return True
     
     
    @@ -285,12 +325,6 @@ def abi_type_of(lll_typ):
             raise CompilerPanic(f"Unrecognized type {lll_typ}")
     
     
    -# utility function, constructs an LLL tuple out of a list of LLL nodes
    -def lll_tuple_from_args(args):
    -    typ = TupleType([x.typ for x in args])
    -    return LLLnode.from_list(["multi"] + [x for x in args], typ=typ)
    -
    -
     # the new type system
     # TODO consider moving these into properties of the type itself
     def abi_type_of2(t: vy.BasePrimitive) -> ABIType:
    @@ -317,14 +351,6 @@ def abi_type_of2(t: vy.BasePrimitive) -> ABIType:
         raise CompilerPanic(f"Unrecognized type {t}")
     
     
    -# there are a lot of places in the calling convention where a tuple
    -# must be passed, so here's a convenience function for that.
    -def ensure_tuple(abi_typ):
    -    if not abi_typ.is_tuple():
    -        return ABI_Tuple([abi_typ])
    -    return abi_typ
    -
    -
     # turn an lll node into a list, based on its type.
     def o_list(lll_node, pos=None):
         lll_t = lll_node.typ
    @@ -338,7 +364,7 @@ def o_list(lll_node, pos=None):
                     else [LLLnode.from_list(i, "uint256") for i in range(lll_t.count)]
                 )
     
    -            ret = [add_variable_offset(lll_node, k, pos, array_bounds_check=False) for k in ks]
    +            ret = [get_element_ptr(lll_node, k, pos, array_bounds_check=False) for k in ks]
             return ret
         else:
             return [lll_node]
    @@ -382,16 +408,38 @@ def abi_encode(dst, lll_node, pos=None, bufsz=None, returns_len=False):
         if bufsz is not None and bufsz < 32 * size_bound:
             raise CompilerPanic("buffer provided to abi_encode not large enough")
     
    +    # fastpath: if there is no dynamic data, we can optimize the
    +    # encoding by using make_setter, since our memory encoding happens
    +    # to be identical to the ABI encoding.
    +    if not parent_abi_t.is_dynamic():
    +        # cast the output buffer to something that make_setter accepts
    +        dst = LLLnode(dst, typ=lll_node.typ, location="memory")
    +        lll_ret = ["seq", make_setter(dst, lll_node, "memory", pos)]
    +        if returns_len:
    +            lll_ret.append(parent_abi_t.embedded_static_size())
    +        return LLLnode.from_list(lll_ret, pos=pos, annotation=f"abi_encode {lll_node.typ}")
    +
         lll_ret = ["seq"]
    +
    +    # contains some computation, we need to only do it once.
    +    if lll_node.is_complex_lll:
    +        to_encode = LLLnode.from_list(
    +            "to_encode", typ=lll_node.typ, location=lll_node.location, encoding=lll_node.encoding
    +        )
    +    else:
    +        to_encode = lll_node
    +
         dyn_ofst = "dyn_ofst"  # current offset in the dynamic section
         dst_begin = "dst"  # pointer to beginning of buffer
         dst_loc = "dst_loc"  # pointer to write location in static section
    -    os = o_list(lll_node, pos=pos)
    +    os = o_list(to_encode, pos=pos)
     
         for i, o in enumerate(os):
             abi_t = abi_type_of(o.typ)
     
    -        if parent_abi_t.is_tuple():
    +        if parent_abi_t.is_complex_type():
    +            # TODO optimize: special case where there is only one dynamic
    +            # member, the location is statically known.
                 if abi_t.is_dynamic():
                     lll_ret.append(["mstore", dst_loc, dyn_ofst])
                     # recurse
    @@ -408,9 +456,11 @@ def abi_encode(dst, lll_node, pos=None, bufsz=None, returns_len=False):
     
             elif isinstance(o.typ, BaseType):
                 d = LLLnode(dst_loc, typ=o.typ, location="memory")
    +            # call into make_setter routine
                 lll_ret.append(make_setter(d, o, location=d.location, pos=pos))
             elif isinstance(o.typ, ByteArrayLike):
                 d = LLLnode.from_list(dst_loc, typ=o.typ, location="memory")
    +            # call into make_setter routinme
                 lll_ret.append(["seq", make_setter(d, o, location=d.location, pos=pos), zero_pad(d)])
             else:
                 raise CompilerPanic(f"unreachable type: {o.typ}")
    @@ -425,7 +475,7 @@ def abi_encode(dst, lll_node, pos=None, bufsz=None, returns_len=False):
         if returns_len:
             if not parent_abi_t.is_dynamic():
                 lll_ret.append(parent_abi_t.embedded_static_size())
    -        elif parent_abi_t.is_tuple():
    +        elif parent_abi_t.is_complex_type():
                 lll_ret.append("dyn_ofst")
             elif isinstance(lll_node.typ, ByteArrayLike):
                 # for abi purposes, return zero-padded length
    @@ -434,37 +484,53 @@ def abi_encode(dst, lll_node, pos=None, bufsz=None, returns_len=False):
             else:
                 raise CompilerPanic("unknown type {lll_node.typ}")
     
    -    if not (parent_abi_t.is_dynamic() and parent_abi_t.is_tuple()):
    +    if not (parent_abi_t.is_dynamic() and parent_abi_t.is_complex_type()):
             pass  # optimize out dyn_ofst allocation if we don't need it
         else:
             dyn_section_start = parent_abi_t.static_size()
             lll_ret = ["with", "dyn_ofst", dyn_section_start, lll_ret]
     
         lll_ret = ["with", dst_begin, dst, ["with", dst_loc, dst_begin, lll_ret]]
     
    -    return LLLnode.from_list(lll_ret, pos=pos)
    +    if lll_node.is_complex_lll:
    +        lll_ret = ["with", to_encode, lll_node, lll_ret]
    +
    +    return LLLnode.from_list(lll_ret, pos=pos, annotation=f"abi_encode {lll_node.typ}")
     
     
     # lll_node is the destination LLL item, src is the input buffer.
     # recursively copy the buffer items into lll_node, based on its type.
     # src: pointer to beginning of buffer
     # src_loc: pointer to read location in static section
    -def abi_decode(lll_node, src, pos=None):
    +def abi_decode(lll_node, src, clamp=True, pos=None):
         os = o_list(lll_node, pos=pos)
         lll_ret = ["seq"]
         parent_abi_t = abi_type_of(lll_node.typ)
         for i, o in enumerate(os):
             abi_t = abi_type_of(o.typ)
             src_loc = LLLnode("src_loc", typ=o.typ, location=src.location)
    -        if parent_abi_t.is_tuple():
    +        if parent_abi_t.is_complex_type():
                 if abi_t.is_dynamic():
    +                # TODO optimize: special case where there is only one dynamic
    +                # member, the location is statically known.
                     child_loc = ["add", "src", unwrap_location(src_loc)]
                     child_loc = LLLnode.from_list(child_loc, typ=o.typ, location=src.location)
                 else:
                     child_loc = src_loc
                 # descend into the child tuple
    -            lll_ret.append(abi_decode(o, child_loc, pos=pos))
    +            lll_ret.append(abi_decode(o, child_loc, clamp=clamp, pos=pos))
    +
             else:
    +
    +            if clamp and _needs_clamp(o.typ, Encoding.ABI):
    +                src_loc = LLLnode.from_list(
    +                    ["with", "src_loc", src_loc, ["seq", clamp_basetype(src_loc), src_loc]],
    +                    typ=src_loc.typ,
    +                    location=src_loc.location,
    +                )
    +            else:
    +                pass
    +
                 lll_ret.append(make_setter(o, src_loc, location=o.location, pos=pos))
     
             if i + 1 == len(os):
    @@ -476,50 +542,3 @@ def abi_decode(lll_node, src, pos=None):
         lll_ret = ["with", "src", src, ["with", "src_loc", "src", lll_ret]]
     
         return lll_ret
    -
    -
    -def _add_ofst(loc, ofst):
    -    if isinstance(loc.value, int):
    -        return LLLnode(loc.value + ofst)
    -    return ["add", loc, ofst]
    -
    -
    -# decode a buffer containing abi-encoded data structure in place
    -# for dynamical data, a layer of indirection will be
    -# added for every level of nesting. for instance,
    -# `lazy_abi_decode(<int128>, <320>)`
    -# might return (mload 320),
    -# whereas
    -# `lazy_abi_decode(<(int128,bytes)>, <320>)`
    -# might return
    -# (multi
    -#   (mload 320/*int128*/)
    -#   (mload (add 320/*buf start*/ (mload 352/*ofst loc*/))))
    -# thought of the day: it might be nice to have an argument like `sanitize`
    -# which will add well-formedness checks (clamps) for all inputs.
    -def lazy_abi_decode(typ, src, pos=None):
    -    if isinstance(typ, (ListType, TupleLike)):
    -        if isinstance(typ, TupleLike):
    -            ts = typ.tuple_members()
    -        else:
    -            ts = [typ.subtyp for _ in range(typ.count)]
    -        ofst = 0
    -        os = []
    -        for t in ts:
    -            child_abi_t = abi_type_of(t)
    -            loc = _add_ofst(src, ofst)
    -            if child_abi_t.is_dynamic():
    -                # load the offset word, which is the
    -                # (location-independent) offset from the start of the
    -                # src buffer.
    -                dyn_ofst = unwrap_location(ofst)
    -                loc = _add_ofst(src, dyn_ofst)
    -            os.append(lazy_abi_decode(t, loc, pos))
    -            ofst += child_abi_t.embedded_static_size()
    -
    -        return LLLnode.from_list(["multi"] + os, typ=typ, pos=pos)
    -
    -    elif isinstance(typ, (BaseType, ByteArrayLike)):
    -        return unwrap_location(src)
    -    else:
    -        raise CompilerPanic(f"unknown type for lazy_abi_decode {typ}")
    
  • vyper/old_codegen/arg_clamps.py+27 141 modified
    @@ -1,155 +1,41 @@
    -import functools
    -import uuid
    -
     from vyper.evm.opcodes import version_check
    -from vyper.old_codegen.lll_node import LLLnode
    -from vyper.old_codegen.types.types import (
    -    ByteArrayLike,
    -    ListType,
    -    get_size_of_type,
    -    is_base_type,
    -)
    -from vyper.utils import MemoryPositions
    -
    -
    -def _mk_calldatacopy_copier(pos, sz, mempos):
    -    return ["calldatacopy", mempos, ["add", 4, pos], sz]
    -
    -
    -def _mk_codecopy_copier(pos, sz, mempos):
    -    return ["codecopy", mempos, ["add", "~codelen", pos], sz]
    -
     
    -def make_arg_clamper(datapos, mempos, typ, is_init=False):
    -    """
    -    Clamps argument to type limits.
    +# TODO this whole module should be replaced with parser_utils.clamp_basetype
     
    -    Arguments
    -    ---------
    -    datapos : int | LLLnode
    -        Calldata offset of the value being clamped
    -    mempos : int | LLLnode
    -        Memory offset that the value is stored at during clamping
    -    typ : vyper.types.types.BaseType
    -        Type of the value
    -    is_init : bool, optional
    -        Boolean indicating if we are generating init bytecode
     
    -    Returns
    -    -------
    -    LLLnode
    -        Arg clamper LLL
    -    """
    -
    -    if not is_init:
    -        data_decl = ["calldataload", ["add", 4, datapos]]
    -        copier = functools.partial(_mk_calldatacopy_copier, mempos=mempos)
    -    else:
    -        data_decl = ["codeload", ["add", "~codelen", datapos]]
    -        copier = functools.partial(_mk_codecopy_copier, mempos=mempos)
    -    # Numbers: make sure they're in range
    -    if is_base_type(typ, "int128"):
    -        return LLLnode.from_list(
    -            int128_clamp(data_decl), typ=typ, annotation="checking int128 input"
    -        )
    -    # Booleans: make sure they're zero or one
    -    elif is_base_type(typ, "bool"):
    -        if version_check(begin="constantinople"):
    -            lll = ["assert", ["iszero", ["shr", 1, data_decl]]]
    -        else:
    -            lll = ["uclamplt", data_decl, 2]
    -        return LLLnode.from_list(lll, typ=typ, annotation="checking bool input")
    -    # Addresses: make sure they're in range
    -    elif is_base_type(typ, "address"):
    -        return LLLnode.from_list(
    -            address_clamp(data_decl), typ=typ, annotation="checking address input"
    -        )
    -    # Bytes: make sure they have the right size
    -    elif isinstance(typ, ByteArrayLike):
    -        return LLLnode.from_list(
    -            [
    -                "seq",
    -                copier(data_decl, 32 + typ.maxlen),
    -                ["assert", ["le", ["calldataload", ["add", 4, data_decl]], typ.maxlen]],
    -            ],
    -            typ=None,
    -            annotation="checking bytearray input",
    -        )
    -    # Lists: recurse
    -    elif isinstance(typ, ListType):
    -        if typ.count > 5 or (type(datapos) is list and type(mempos) is list):
    -            # find ultimate base type
    -            subtype = typ.subtype
    -            while hasattr(subtype, "subtype"):
    -                subtype = subtype.subtype
    -
    -            # make arg clamper for the base type
    -            offset = MemoryPositions.FREE_LOOP_INDEX
    -            clamper = make_arg_clamper(
    -                ["add", datapos, ["mload", offset]],
    -                ["add", mempos, ["mload", offset]],
    -                subtype,
    -                is_init,
    -            )
    -            if clamper.value == "pass":
    -                # no point looping if the base type doesn't require clamping
    -                return clamper
    +def _shr(x, bits):
    +    if version_check(begin="constantinople"):
    +        return ["shr", bits, x]
    +    return ["div", x, ["exp", 2, bits]]
     
    -            # loop the entire array at once, even if it's multidimensional
    -            type_size = get_size_of_type(typ)
    -            i_incr = get_size_of_type(subtype) * 32
     
    -            mem_to = type_size * 32
    -            loop_label = f"_check_list_loop_{str(uuid.uuid4())}"
    +def _sar(x, bits):
    +    if version_check(begin="constantinople"):
    +        return ["sar", bits, x]
     
    -            lll_node = [
    -                ["mstore", offset, 0],  # init loop
    -                ["label", loop_label],
    -                clamper,
    -                ["mstore", offset, ["add", ["mload", offset], i_incr]],
    -                ["if", ["lt", ["mload", offset], mem_to], ["goto", loop_label]],
    -            ]
    -        else:
    -            lll_node = []
    -            for i in range(typ.count):
    -                offset = get_size_of_type(typ.subtype) * 32 * i
    -                lll_node.append(
    -                    make_arg_clamper(datapos + offset, mempos + offset, typ.subtype, is_init)
    -                )
    -        return LLLnode.from_list(["seq"] + lll_node, typ=None, annotation="checking list input")
    -    # Otherwise don't make any checks
    -    else:
    -        return LLLnode.from_list("pass")
    +    # emulate for older arches. keep in mind note from EIP 145:
    +    # This is not equivalent to PUSH1 2 EXP SDIV, since it rounds
    +    # differently. See SDIV(-1, 2) == 0, while SAR(-1, 1) == -1.
    +    return ["sdiv", ["add", ["slt", x, 0], x], ["exp", 2, bits]]
     
     
     def address_clamp(lll_node):
    -    if version_check(begin="constantinople"):
    -        return ["assert", ["iszero", ["shr", 160, lll_node]]]
    -    else:
    -        return ["uclamplt", lll_node, ["mload", MemoryPositions.ADDRSIZE]]
    +    return ["assert", ["iszero", _shr(lll_node, 160)]]
     
     
     def int128_clamp(lll_node):
    -    if version_check(begin="constantinople"):
    -        return [
    -            "with",
    +    return [
    +        "with",
    +        "_val",
    +        lll_node,
    +        [
    +            "seq",
    +            # if _val is in bounds,
    +            # _val >>> 127 == 0 for positive _val
    +            # _val >>> 127 == -1 for negative _val
    +            # -1 and 0 are the only numbers which are unchanged by sar,
    +            # so sar'ing (_val>>>127) one more bit should leave it unchanged.
    +            ["assert", ["eq", _sar("_val", 128), _sar("_val", 127)]],
                 "_val",
    -            lll_node,
    -            [
    -                "seq",
    -                # if _val is in bounds,
    -                # _val >>> 127 == 0 for positive _val
    -                # _val >>> 127 == -1 for negative _val
    -                # -1 and 0 are the only numbers which are unchanged by sar,
    -                # so sar'ing (_val>>>127) one more bit should leave it unchanged.
    -                ["assert", ["eq", ["sar", 128, "_val"], ["sar", 127, "_val"]]],
    -                "_val",
    -            ],
    -        ]
    -    else:
    -        return [
    -            "clamp",
    -            ["mload", MemoryPositions.MIN_INT128],
    -            lll_node,
    -            ["mload", MemoryPositions.MAX_INT128],
    -        ]
    +        ],
    +    ]
    
  • vyper/old_codegen/context.py+60 7 modified
    @@ -3,7 +3,7 @@
     
     from vyper.ast import VyperNode
     from vyper.ast.signatures.function_signature import VariableRecord
    -from vyper.exceptions import CompilerPanic
    +from vyper.exceptions import CompilerPanic, FunctionDeclarationException
     from vyper.old_codegen.types import NodeType, get_size_of_type
     
     
    @@ -25,7 +25,7 @@ def __init__(
             constancy=Constancy.Mutable,
             is_internal=False,
             is_payable=False,
    -        method_id="",
    +        # method_id="",
             sig=None,
         ):
             # In-memory variables, in the form (name, memory location, type)
    @@ -52,7 +52,7 @@ def __init__(
             self.callback_ptr = None
             self.is_internal = is_internal
             # method_id of current function
    -        self.method_id = method_id
    +        # self.method_id = method_id
             # store global context
             self.global_ctx = global_ctx
             # full function signature
    @@ -64,13 +64,24 @@ def __init__(
             # Not intended to be accessed directly
             self.memory_allocator = memory_allocator
     
    +        self._callee_frame_sizes = []
    +
             # Intermented values, used for internal IDs
             self._internal_var_iter = 0
             self._scope_id_iter = 0
     
         def is_constant(self):
             return self.constancy is Constancy.Constant or self.in_assertion or self.in_range_expr
     
    +    def register_callee(self, frame_size):
    +        self._callee_frame_sizes.append(frame_size)
    +
    +    @property
    +    def max_callee_frame_size(self):
    +        if len(self._callee_frame_sizes) == 0:
    +            return 0
    +        return max(self._callee_frame_sizes)
    +
         #
         # Context Managers
         # - Context managers are used to ensure proper wrapping of scopes and context states.
    @@ -128,7 +139,9 @@ def block_scope(self):
             # Remove block scopes
             self._scopes.remove(scope_id)
     
    -    def _new_variable(self, name: str, typ: NodeType, var_size: int, is_internal: bool) -> int:
    +    def _new_variable(
    +        self, name: str, typ: NodeType, var_size: int, is_internal: bool, is_mutable: bool = True
    +    ) -> int:
             if is_internal:
                 var_pos = self.memory_allocator.expand_memory(var_size)
             else:
    @@ -137,13 +150,15 @@ def _new_variable(self, name: str, typ: NodeType, var_size: int, is_internal: bo
                 name=name,
                 pos=var_pos,
                 typ=typ,
    -            mutable=True,
    +            mutable=is_mutable,
                 blockscopes=self._scopes.copy(),
                 is_internal=is_internal,
             )
             return var_pos
     
    -    def new_variable(self, name: str, typ: NodeType, pos: VyperNode = None) -> int:
    +    def new_variable(
    +        self, name: str, typ: NodeType, pos: VyperNode = None, is_mutable: bool = True
    +    ) -> int:
             """
             Allocate memory for a user-defined variable.
     
    @@ -168,8 +183,9 @@ def new_variable(self, name: str, typ: NodeType, pos: VyperNode = None) -> int:
                 var_size = typ.size_in_bytes  # type: ignore
             else:
                 var_size = 32 * get_size_of_type(typ)
    -        return self._new_variable(name, typ, var_size, False)
    +        return self._new_variable(name, typ, var_size, False, is_mutable=is_mutable)
     
    +    # do we ever allocate immutable internal variables?
         def new_internal_variable(self, typ: NodeType) -> int:
             """
             Allocate memory for an internal variable.
    @@ -199,6 +215,43 @@ def new_internal_variable(self, typ: NodeType) -> int:
         def parse_type(self, ast_node, location):
             return self.global_ctx.parse_type(ast_node, location)
     
    +    def lookup_var(self, varname):
    +        return self.vars[varname]
    +
    +    def lookup_internal_function(self, method_name, args_lll):
    +        # TODO is this the right module for me?
    +        """
    +        Using a list of args, find the internal method to use, and
    +        the kwargs which need to be filled in by the compiler
    +        """
    +
    +        def _check(cond, s="Unreachable"):
    +            if not cond:
    +                raise CompilerPanic(s)
    +
    +        sig = self.sigs["self"].get(method_name, None)
    +        if sig is None:
    +            raise FunctionDeclarationException(
    +                "Function does not exist or has not been declared yet "
    +                "(reminder: functions cannot call functions later in code "
    +                f"than themselves): {method_name}"
    +            )
    +
    +        _check(sig.internal)  # sanity check
    +        # should have been caught during type checking, sanity check anyway
    +        _check(len(sig.base_args) <= len(args_lll) <= len(sig.args))
    +
    +        # more sanity check, that the types match
    +        # _check(all(l.typ == r.typ for (l, r) in zip(args_lll, sig.args))
    +
    +        num_provided_kwargs = len(args_lll) - len(sig.base_args)
    +        num_kwargs = len(sig.default_args)
    +        kwargs_needed = num_kwargs - num_provided_kwargs
    +
    +        kw_vals = list(sig.default_values.values())[:kwargs_needed]
    +
    +        return sig, kw_vals
    +
         # Pretty print constancy for error messages
         def pp_constancy(self):
             if self.in_assertion:
    
  • vyper/old_codegen/events.py+5 2 modified
    @@ -6,12 +6,15 @@
         abi_encode,
         abi_type_of,
         abi_type_of2,
    -    lll_tuple_from_args,
     )
     from vyper.old_codegen.context import Context
     from vyper.old_codegen.keccak256_helper import keccak256_helper
     from vyper.old_codegen.lll_node import LLLnode
    -from vyper.old_codegen.parser_utils import getpos, unwrap_location
    +from vyper.old_codegen.parser_utils import (
    +    getpos,
    +    lll_tuple_from_args,
    +    unwrap_location,
    +)
     from vyper.old_codegen.types.types import (
         BaseType,
         ByteArrayLike,
    
  • vyper/old_codegen/expr.py+33 109 modified
    @@ -15,10 +15,10 @@
     from vyper.old_codegen.keccak256_helper import keccak256_helper
     from vyper.old_codegen.lll_node import LLLnode
     from vyper.old_codegen.parser_utils import (
    -    add_variable_offset,
    +    get_element_ptr,
         get_number_as_fraction,
         getpos,
    -    make_byte_array_copier,
    +    load_op,
         make_setter,
         unwrap_location,
     )
    @@ -202,7 +202,7 @@ def __init__(self, node, context):
     
             self.lll_node = fn()
             if self.lll_node is None:
    -            raise TypeCheckFailure(f"{type(node).__name__} node did not produce LLL")
    +            raise TypeCheckFailure(f"{type(node).__name__} node did not produce LLL. {self.expr}")
     
         def parse_Int(self):
             # Literal (mostly likely) becomes int256
    @@ -298,6 +298,7 @@ def parse_Name(self):
                     var.pos,
                     typ=var.typ,
                     location=var.location,  # either 'memory' or 'calldata' storage is handled above.
    +                encoding=var.encoding,
                     pos=getpos(self.expr),
                     annotation=self.expr.id,
                     mutable=var.mutable,
    @@ -411,42 +412,29 @@ def parse_Attribute(self):
                 if isinstance(sub.typ, InterfaceType):
                     return sub
                 if isinstance(sub.typ, StructType) and self.expr.attr in sub.typ.members:
    -                return add_variable_offset(sub, self.expr.attr, pos=getpos(self.expr))
    +                return get_element_ptr(sub, self.expr.attr, pos=getpos(self.expr))
     
         def parse_Subscript(self):
             sub = Expr.parse_variable_location(self.expr.value, self.context)
    -        if isinstance(sub.typ, (MappingType, ListType)):
    +
    +        if isinstance(sub.typ, MappingType):
    +            # TODO sanity check we are in a self.my_map[i] situation
                 index = Expr.parse_value_expr(self.expr.slice.value, self.context)
    -            if isinstance(index.typ, ByteArrayLike) and index.args[0].location == "storage":
    -                # Special case - if the key value is a bytes-array type located in
    -                # storage, we have to copy it to memory prior to calculating the hash
    -                placeholder = self.context.new_internal_variable(index.typ)
    -                placeholder_node = LLLnode.from_list(placeholder, typ=index.typ, location="memory")
    -                copier = make_byte_array_copier(
    -                    placeholder_node,
    -                    LLLnode.from_list(index.args[0], typ=index.typ, location="storage"),
    -                )
    -                return LLLnode.from_list(
    -                    [
    -                        "seq",
    -                        copier,
    -                        [
    -                            "sha3_64",
    -                            sub,
    -                            ["sha3", ["add", placeholder, 32], ["mload", placeholder]],
    -                        ],
    -                    ],
    -                    typ=sub.typ.valuetype,
    -                    pos=getpos(self.expr),
    -                    location="storage",
    -                )
    +            if isinstance(index.typ, ByteArrayLike):
    +                # special case,
    +                # we have to hash the key to get a storage location
    +                index = keccak256_helper(self.expr.slice.value, index.args, None, self.context)
    +
    +        elif isinstance(sub.typ, ListType):
    +            index = Expr.parse_value_expr(self.expr.slice.value, self.context)
    +
             elif isinstance(sub.typ, TupleType):
                 index = self.expr.slice.value.n
                 if not 0 <= index < len(sub.typ.members):
                     return
             else:
                 return
    -        lll_node = add_variable_offset(sub, index, pos=getpos(self.expr))
    +        lll_node = get_element_ptr(sub, index, pos=getpos(self.expr))
             lll_node.mutable = sub.mutable
             return lll_node
     
    @@ -780,7 +768,8 @@ def build_in_comparator(self):
                 # for `not in`, invert the result
                 compare_sequence = ["iszero", compare_sequence]
     
    -        return LLLnode.from_list(compare_sequence, typ="bool", annotation="in comparator")
    +        annotation = self.expr.get("node_source_code")
    +        return LLLnode.from_list(compare_sequence, typ="bool", annotation=annotation)
     
         @staticmethod
         def _signed_to_unsigned_comparision_op(op):
    @@ -849,10 +838,11 @@ def parse_Compare(self):
                 else:
     
                     def load_bytearray(side):
    -                    if side.location == "memory":
    -                        return ["mload", ["add", 32, side]]
    -                    elif side.location == "storage":
    +                    if side.location == "storage":
                             return ["sload", ["add", 1, side]]
    +                    else:
    +                        load = load_op(side.location)
    +                        return [load, ["add", 32, side]]
     
                     return LLLnode.from_list(
                         [op, load_bytearray(left), load_bytearray(right)],
    @@ -978,30 +968,24 @@ def parse_Call(self):
                 and isinstance(self.expr.func.value, vy_ast.Name)
                 and self.expr.func.value.id == "self"
             ):  # noqa: E501
    -            return self_call.make_call(self.expr, self.context)
    +            return self_call.lll_for_self_call(self.expr, self.context)
             else:
    -            return external_call.make_external_call(self.expr, self.context)
    +            return external_call.lll_for_external_call(self.expr, self.context)
     
         def parse_List(self):
    -        call_lll, multi_lll = parse_sequence(self.expr, self.expr.elements, self.context)
    +        multi_lll = [Expr(x, self.context).lll_node for x in self.expr.elements]
    +        # TODO this type inference is wrong. instead should use
    +        # parse_type(canonical_type_of(self.expr._metadata["type"]))
             out_type = next((i.typ for i in multi_lll if not i.typ.is_literal), multi_lll[0].typ)
             typ = ListType(out_type, len(self.expr.elements), is_literal=True)
             multi_lll = LLLnode.from_list(["multi"] + multi_lll, typ=typ, pos=getpos(self.expr))
    -        if not call_lll:
    -            return multi_lll
    -
    -        lll_node = ["seq_unchecked"] + call_lll + [multi_lll]
    -        return LLLnode.from_list(lll_node, typ=typ, pos=getpos(self.expr))
    +        return multi_lll
     
         def parse_Tuple(self):
    -        call_lll, multi_lll = parse_sequence(self.expr, self.expr.elements, self.context)
    -        typ = TupleType([x.typ for x in multi_lll], is_literal=True)
    -        multi_lll = LLLnode.from_list(["multi"] + multi_lll, typ=typ, pos=getpos(self.expr))
    -        if not call_lll:
    -            return multi_lll
    -
    -        lll_node = ["seq_unchecked"] + call_lll + [multi_lll]
    -        return LLLnode.from_list(lll_node, typ=typ, pos=getpos(self.expr))
    +        tuple_elements = [Expr(x, self.context).lll_node for x in self.expr.elements]
    +        typ = TupleType([x.typ for x in tuple_elements], is_literal=True)
    +        multi_lll = LLLnode.from_list(["multi"] + tuple_elements, typ=typ, pos=getpos(self.expr))
    +        return multi_lll
     
         @staticmethod
         def struct_literals(expr, name, context):
    @@ -1033,63 +1017,3 @@ def parse_variable_location(cls, expr, context):
             if not o.location:
                 raise StructureException("Looking for a variable location, instead got a value", expr)
             return o
    -
    -
    -def parse_sequence(base_node, elements, context):
    -    """
    -    Generate an LLL node from a sequence of Vyper AST nodes, such as values inside a
    -    list/tuple or arguments inside a call.
    -
    -    Arguments
    -    ---------
    -    base_node : VyperNode
    -        Parent node which contains the sequence being parsed.
    -    elements : List[VyperNode]
    -        A list of nodes within the sequence.
    -    context : Context
    -        Currently active local context.
    -
    -    Returns
    -    -------
    -    List[LLLNode]
    -        LLL nodes that must execute prior to generating the actual sequence in order to
    -        avoid memory corruption issues. This list may be empty, depending on the values
    -        within `elements`.
    -    List[LLLNode]
    -        LLL nodes which collectively represent `elements`.
    -    """
    -    init_lll = []
    -    sequence_lll = []
    -    for node in elements:
    -        if isinstance(node, vy_ast.List):
    -            # for nested lists, ensure the init LLL is also processed before the values
    -            init, seq = parse_sequence(node, node.elements, context)
    -            init_lll.extend(init)
    -            out_type = next((i.typ for i in seq if not i.typ.is_literal), seq[0].typ)
    -            typ = ListType(out_type, len(node.elements), is_literal=True)
    -            multi_lll = LLLnode.from_list(["multi"] + seq, typ=typ, pos=getpos(node))
    -            sequence_lll.append(multi_lll)
    -            continue
    -
    -        lll_node = Expr(node, context).lll_node
    -        if isinstance(node, vy_ast.Call) or (
    -            isinstance(node, vy_ast.Subscript) and isinstance(node.value, vy_ast.Call)
    -        ):
    -            # nodes which potentially create their own internal memory variables, and so must
    -            # be parsed prior to generating the final sequence to avoid memory corruption
    -            target = LLLnode.from_list(
    -                context.new_internal_variable(lll_node.typ),
    -                typ=lll_node.typ,
    -                location="memory",
    -                pos=getpos(base_node),
    -            )
    -            init_lll.append(make_setter(target, lll_node, "memory", pos=getpos(base_node)))
    -            sequence_lll.append(
    -                LLLnode.from_list(
    -                    target, typ=lll_node.typ, pos=getpos(base_node), location="memory"
    -                ),
    -            )
    -        else:
    -            sequence_lll.append(lll_node)
    -
    -    return init_lll, sequence_lll
    
  • vyper/old_codegen/external_call.py+186 187 modified
    @@ -1,164 +1,188 @@
    +import vyper.utils as util
     from vyper import ast as vy_ast
     from vyper.exceptions import (
         StateAccessViolation,
         StructureException,
         TypeCheckFailure,
     )
    -from vyper.old_codegen.abi import abi_decode
    -from vyper.old_codegen.lll_node import LLLnode
    +from vyper.old_codegen.abi import abi_encode, abi_type_of
    +from vyper.old_codegen.lll_node import Encoding, LLLnode
     from vyper.old_codegen.parser_utils import (
    +    calculate_type_for_external_return,
    +    get_element_ptr,
         getpos,
    -    pack_arguments,
         unwrap_location,
     )
     from vyper.old_codegen.types import (
    -    BaseType,
    -    ByteArrayLike,
    -    ListType,
    -    TupleLike,
    -    get_size_of_type,
    -    get_static_size_of_type,
    -    has_dynamic_data,
    +    TupleType,
    +    canonicalize_type,
    +    get_type_for_exact_size,
     )
    +from vyper.old_codegen.types.check import check_assign
     
     
    -def external_call(node, context, interface_name, contract_address, pos, value=None, gas=None):
    -    from vyper.old_codegen.expr import Expr
    +def _pack_arguments(contract_sig, args, context, pos):
    +    # abi encoding just treats all args as a big tuple
    +    args_tuple_t = TupleType([x.typ for x in args])
    +    args_as_tuple = LLLnode.from_list(["multi"] + [x for x in args], typ=args_tuple_t)
    +    args_abi_t = abi_type_of(args_tuple_t)
    +
    +    # sanity typecheck - make sure the arguments can be assigned
    +    dst_tuple_t = TupleType([arg.typ for arg in contract_sig.args][: len(args)])
    +    _tmp = LLLnode("fake node", location="memory", typ=dst_tuple_t)
    +    check_assign(_tmp, args_as_tuple, pos)
    +
    +    if contract_sig.return_type is not None:
    +        return_abi_t = abi_type_of(calculate_type_for_external_return(contract_sig.return_type))
    +
    +        # we use the same buffer for args and returndata,
    +        # so allocate enough space here for the returndata too.
    +        buflen = max(args_abi_t.size_bound(), return_abi_t.size_bound())
    +    else:
    +        buflen = args_abi_t.size_bound()
    +
    +    buflen += 32  # padding for the method id
    +
    +    buf_t = get_type_for_exact_size(buflen)
    +    buf = context.new_internal_variable(buf_t)
    +
    +    args_ofst = buf + 28
    +    args_len = args_abi_t.size_bound() + 4
    +
    +    abi_signature = contract_sig.name + canonicalize_type(dst_tuple_t)
    +
    +    # layout:
    +    # 32 bytes                 | args
    +    # 0x..00<method_id_4bytes> | args
    +    # the reason for the left padding is just so the alignment is easier.
    +    # if we were only targeting constantinople, we could align
    +    # to buf (and also keep code size small) by using
    +    # (mstore buf (shl signature.method_id 224))
    +    mstore_method_id = [["mstore", buf, util.abi_method_id(abi_signature)]]
    +
    +    if len(args) == 0:
    +        encode_args = ["pass"]
    +    else:
    +        encode_args = abi_encode(buf + 32, args_as_tuple, pos)
    +
    +    return buf, mstore_method_id + [encode_args], args_ofst, args_len
    +
    +
    +def _returndata_encoding(contract_sig):
    +    if contract_sig.is_from_json:
    +        return Encoding.JSON_ABI
    +    return Encoding.ABI
    +
    +
    +def _unpack_returndata(buf, contract_sig, context, pos):
    +    return_t = contract_sig.return_type
    +    if return_t is None:
    +        return ["pass"], 0, 0
    +
    +    return_t = calculate_type_for_external_return(return_t)
    +    # if the abi signature has a different type than
    +    # the vyper type, we need to wrap and unwrap the type
    +    # so that the ABI decoding works correctly
    +    should_unwrap_abi_tuple = return_t != contract_sig.return_type
    +
    +    abi_return_t = abi_type_of(return_t)
    +
    +    min_return_size = abi_return_t.min_size()
    +    max_return_size = abi_return_t.size_bound()
    +    assert 0 < min_return_size <= max_return_size
    +
    +    ret_ofst = buf
    +    ret_len = max_return_size
    +
    +    # revert when returndatasize is not in bounds
    +    ret = []
    +    # runtime: min_return_size <= returndatasize
    +    # TODO move the -1 optimization to LLL optimizer
    +    ret += [["assert", ["gt", "returndatasize", min_return_size - 1]]]
    +
    +    # add as the last LLLnode a pointer to the return data structure
    +
    +    # the return type has been wrapped by the calling contract;
    +    # unwrap it so downstream code isn't confused.
    +    # basically this expands to buf+32 if the return type has been wrapped
    +    # in a tuple AND its ABI type is dynamic.
    +    # in most cases, this simply will evaluate to ret.
    +    # in the special case where the return type has been wrapped
    +    # in a tuple AND its ABI type is dynamic, it expands to buf+32.
    +    buf = LLLnode(buf, typ=return_t, encoding=_returndata_encoding(contract_sig), location="memory")
    +
    +    if should_unwrap_abi_tuple:
    +        buf = get_element_ptr(buf, 0, pos=None, array_bounds_check=False)
    +
    +    ret += [buf]
    +
    +    return ret, ret_ofst, ret_len
    +
    +
    +def _external_call_helper(
    +    contract_address, contract_sig, args_lll, context, pos=None, value=None, gas=None
    +):
     
         if value is None:
             value = 0
         if gas is None:
             gas = "gas"
     
    -    method_name = node.func.attr
    -    sig = context.sigs[interface_name][method_name]
    -    inargs, inargsize, _ = pack_arguments(
    -        sig,
    -        [Expr(arg, context).lll_node for arg in node.args],
    -        context,
    -        node.func,
    -        is_external_call=True,
    -    )
    -    output_placeholder, output_size, returner = get_external_call_output(sig, context)
    -    sub = ["seq"]
    -    if not output_size:
    -        # if we do not expect return data, check that a contract exists at the target address
    -        # we can omit this when we _do_ expect return data because we later check `returndatasize`
    -        sub.append(["assert", ["extcodesize", contract_address]])
    -    if context.is_constant() and sig.mutability not in ("view", "pure"):
    -        # TODO this can probably go
    +    # sanity check
    +    assert len(contract_sig.args) == len(args_lll)
    +
    +    if context.is_constant() and contract_sig.mutability not in ("view", "pure"):
    +        # TODO is this already done in type checker?
             raise StateAccessViolation(
    -            f"May not call state modifying function '{method_name}' "
    +            f"May not call state modifying function '{contract_sig.name}' "
                 f"within {context.pp_constancy()}.",
    -            node,
    +            pos,
             )
     
    -    if context.is_constant() or sig.mutability in ("view", "pure"):
    -        sub.append(
    -            [
    -                "assert",
    -                [
    -                    "staticcall",
    -                    gas,
    -                    contract_address,
    -                    inargs,
    -                    inargsize,
    -                    output_placeholder,
    -                    output_size,
    -                ],
    -            ]
    -        )
    -    else:
    -        sub.append(
    -            [
    -                "assert",
    -                [
    -                    "call",
    -                    gas,
    -                    contract_address,
    -                    value,
    -                    inargs,
    -                    inargsize,
    -                    output_placeholder,
    -                    output_size,
    -                ],
    -            ]
    -        )
    -    if output_size:
    -        # when return data is expected, revert when the length of `returndatasize` is insufficient
    -        output_type = sig.output_type
    -        if not has_dynamic_data(output_type):
    -            static_output_size = get_static_size_of_type(output_type) * 32
    -            sub.append(["assert", ["gt", "returndatasize", static_output_size - 1]])
    -        else:
    -            if isinstance(output_type, ByteArrayLike):
    -                types_list = (output_type,)
    -            elif isinstance(output_type, TupleLike):
    -                types_list = output_type.tuple_members()
    -            else:
    -                raise
    -
    -            dynamic_checks = []
    -            static_offset = output_placeholder
    -            static_output_size = 0
    -            for typ in types_list:
    -                # ensure length of bytes does not exceed max allowable length for type
    -                if isinstance(typ, ByteArrayLike):
    -                    static_output_size += 32
    -                    # do not perform this check on calls to a JSON interface - we don't know
    -                    # for certain how long the expected data is
    -                    if not sig.is_from_json:
    -                        dynamic_checks.append(
    -                            [
    -                                "assert",
    -                                [
    -                                    "lt",
    -                                    [
    -                                        "mload",
    -                                        ["add", ["mload", static_offset], output_placeholder],
    -                                    ],
    -                                    typ.maxlen + 1,
    -                                ],
    -                            ]
    -                        )
    -                static_offset += get_static_size_of_type(typ) * 32
    -                static_output_size += get_static_size_of_type(typ) * 32
    -
    -            sub.append(["assert", ["gt", "returndatasize", static_output_size - 1]])
    -            sub.extend(dynamic_checks)
    -
    -    sub.extend(returner)
    -
    -    return LLLnode.from_list(sub, typ=sig.output_type, location="memory", pos=getpos(node))
    -
    -
    -def get_external_call_output(sig, context):
    -    if not sig.output_type:
    -        return 0, 0, []
    -    output_placeholder = context.new_internal_variable(typ=sig.output_type)
    -    output_size = get_size_of_type(sig.output_type) * 32
    -    if isinstance(sig.output_type, BaseType):
    -        returner = [0, output_placeholder]
    -    elif isinstance(sig.output_type, ByteArrayLike):
    -        returner = [0, output_placeholder + 32]
    -    elif isinstance(sig.output_type, TupleLike):
    -        # incase of struct we need to decode the output and then return it
    -        returner = ["seq"]
    -        decoded_placeholder = context.new_internal_variable(typ=sig.output_type)
    -        decoded_node = LLLnode(decoded_placeholder, typ=sig.output_type, location="memory")
    -        output_node = LLLnode(output_placeholder, typ=sig.output_type, location="memory")
    -        returner.append(abi_decode(decoded_node, output_node))
    -        returner.extend([0, decoded_placeholder])
    -    elif isinstance(sig.output_type, ListType):
    -        returner = [0, output_placeholder]
    +    sub = ["seq"]
    +
    +    buf, arg_packer, args_ofst, args_len = _pack_arguments(contract_sig, args_lll, context, pos)
    +
    +    ret_unpacker, ret_ofst, ret_len = _unpack_returndata(buf, contract_sig, context, pos)
    +
    +    sub += arg_packer
    +
    +    if contract_sig.return_type is None:
    +        # if we do not expect return data, check that a contract exists at the
    +        # target address. we must perform this check BEFORE the call because
    +        # the contract might selfdestruct. on the other hand we can omit this
    +        # when we _do_ expect return data because we later check
    +        # `returndatasize` (that check works even if the contract
    +        # selfdestructs).
    +        sub.append(["assert", ["extcodesize", contract_address]])
    +
    +    if context.is_constant() or contract_sig.mutability in ("view", "pure"):
    +        call_op = ["staticcall", gas, contract_address, args_ofst, args_len, ret_ofst, ret_len]
         else:
    -        raise TypeCheckFailure(f"Invalid output type: {sig.output_type}")
    -    return output_placeholder, output_size, returner
    +        call_op = ["call", gas, contract_address, value, args_ofst, args_len, ret_ofst, ret_len]
     
    +    sub.append(["assert", call_op])
     
    -def get_external_interface_keywords(stmt_expr, context):
    -    # circular import!
    -    from vyper.old_codegen.expr import Expr
    +    if contract_sig.return_type is not None:
    +        sub += ret_unpacker
    +
    +    ret = LLLnode.from_list(
    +        # set the encoding to ABI here, downstream code will decode and add clampers.
    +        sub,
    +        typ=contract_sig.return_type,
    +        location="memory",
    +        encoding=_returndata_encoding(contract_sig),
    +        pos=pos,
    +    )
    +
    +    return ret
    +
    +
    +# TODO push me up to expr.py
    +def get_gas_and_value(stmt_expr, context):
    +    from vyper.old_codegen.expr import (
    +        Expr,  # TODO rethink this circular import
    +    )
     
         value, gas = None, None
         for kw in stmt_expr.keywords:
    @@ -171,60 +195,35 @@ def get_external_interface_keywords(stmt_expr, context):
         return value, gas
     
     
    -def make_external_call(stmt_expr, context):
    -    # circular import!
    -    from vyper.old_codegen.expr import Expr
    +def lll_for_external_call(stmt_expr, context):
    +    from vyper.old_codegen.expr import (
    +        Expr,  # TODO rethink this circular import
    +    )
     
    -    value, gas = get_external_interface_keywords(stmt_expr, context)
    +    pos = getpos(stmt_expr)
    +    value, gas = get_gas_and_value(stmt_expr, context)
    +    args_lll = [Expr(x, context).lll_node for x in stmt_expr.args]
     
         if isinstance(stmt_expr.func, vy_ast.Attribute) and isinstance(
             stmt_expr.func.value, vy_ast.Call
         ):
    +        # e.g. `Foo(address).bar()`
    +
    +        # sanity check
    +        assert len(stmt_expr.func.value.args) == 1
             contract_name = stmt_expr.func.value.func.id
             contract_address = Expr.parse_value_expr(stmt_expr.func.value.args[0], context)
     
    -        return external_call(
    -            stmt_expr,
    -            context,
    -            contract_name,
    -            contract_address,
    -            pos=getpos(stmt_expr),
    -            value=value,
    -            gas=gas,
    -        )
    -
    -    elif (
    -        isinstance(stmt_expr.func.value, vy_ast.Attribute)
    -        and stmt_expr.func.value.attr in context.sigs
    -    ):  # noqa: E501
    -        contract_name = stmt_expr.func.value.attr
    -        type_ = stmt_expr.func.value._metadata["type"]
    -        var = context.globals[stmt_expr.func.value.attr]
    -        contract_address = unwrap_location(
    -            LLLnode.from_list(
    -                type_.position.position,
    -                typ=var.typ,
    -                location="storage",
    -                pos=getpos(stmt_expr),
    -                annotation="self." + stmt_expr.func.value.attr,
    -            )
    -        )
    -
    -        return external_call(
    -            stmt_expr,
    -            context,
    -            contract_name,
    -            contract_address,
    -            pos=getpos(stmt_expr),
    -            value=value,
    -            gas=gas,
    -        )
    -
         elif (
             isinstance(stmt_expr.func.value, vy_ast.Attribute)
             and stmt_expr.func.value.attr in context.globals
    +        # TODO check for self?
             and hasattr(context.globals[stmt_expr.func.value.attr].typ, "name")
         ):
    +        # e.g. `self.foo.bar()`
    +
    +        # sanity check
    +        assert stmt_expr.func.value.value.id == "self", stmt_expr
     
             contract_name = context.globals[stmt_expr.func.value.attr].typ.name
             type_ = stmt_expr.func.value._metadata["type"]
    @@ -234,20 +233,20 @@ def make_external_call(stmt_expr, context):
                     type_.position.position,
                     typ=var.typ,
                     location="storage",
    -                pos=getpos(stmt_expr),
    +                pos=pos,
                     annotation="self." + stmt_expr.func.value.attr,
                 )
             )
    -
    -        return external_call(
    -            stmt_expr,
    -            context,
    -            contract_name,
    -            contract_address,
    -            pos=getpos(stmt_expr),
    -            value=value,
    -            gas=gas,
    -        )
    -
         else:
    +        # TODO catch this during type checking
             raise StructureException("Unsupported operator.", stmt_expr)
    +
    +    method_name = stmt_expr.func.attr
    +    contract_sig = context.sigs[contract_name][method_name]
    +
    +    ret = _external_call_helper(
    +        contract_address, contract_sig, args_lll, context, pos, value=value, gas=gas,
    +    )
    +    ret.annotation = stmt_expr.get("node_source_code")
    +
    +    return ret
    
  • vyper/old_codegen/function_definitions/common.py+107 0 added
    @@ -0,0 +1,107 @@
    +# can't use from [module] import [object] because it breaks mocks in testing
    +import copy
    +from typing import Dict, Optional, Tuple
    +
    +import vyper.ast as vy_ast
    +from vyper.ast.signatures import FunctionSignature, VariableRecord
    +from vyper.old_codegen.context import Constancy, Context
    +from vyper.old_codegen.function_definitions.external_function import (
    +    generate_lll_for_external_function,
    +)
    +from vyper.old_codegen.function_definitions.internal_function import (
    +    generate_lll_for_internal_function,
    +)
    +from vyper.old_codegen.global_context import GlobalContext
    +from vyper.old_codegen.lll_node import LLLnode
    +from vyper.old_codegen.memory_allocator import MemoryAllocator
    +from vyper.old_codegen.parser_utils import check_single_exit
    +from vyper.utils import MemoryPositions, calc_mem_gas
    +
    +
    +# Is a function the initializer?
    +def is_initializer(code: vy_ast.FunctionDef) -> bool:
    +    return code.name == "__init__"
    +
    +
    +# Is a function the default function?
    +def is_default_func(code: vy_ast.FunctionDef) -> bool:
    +    return code.name == "__default__"
    +
    +
    +def generate_lll_for_function(
    +    code: vy_ast.FunctionDef,
    +    sigs: Dict[str, Dict[str, FunctionSignature]],
    +    global_ctx: GlobalContext,
    +    check_nonpayable: bool,
    +    # CMC 20210921 TODO _vars can probably be removed
    +    _vars: Optional[Dict[str, VariableRecord]] = None,
    +) -> Tuple[LLLnode, int, int]:
    +    """
    +    Parse a function and produce LLL code for the function, includes:
    +        - Signature method if statement
    +        - Argument handling
    +        - Clamping and copying of arguments
    +        - Function body
    +    """
    +    if _vars is None:
    +        _vars = {}  # noqa: F841
    +    sig = FunctionSignature.from_definition(code, sigs=sigs, custom_structs=global_ctx._structs,)
    +
    +    # Validate return statements.
    +    check_single_exit(code)
    +
    +    # in order to statically allocate function frames,
    +    # we codegen functions in two passes.
    +    # one pass is just called for its side effects on the context/memory
    +    # allocator. once that pass is finished, we inspect the context
    +    # to see what the max frame size of any callee in the function was,
    +    # then we run the codegen again with the max frame size as
    +    # the start of the frame for this function.
    +    def _run_pass(memory_allocator=None):
    +        # Create a local (per function) context.
    +        if memory_allocator is None:
    +            memory_allocator = MemoryAllocator()
    +        nonlocal _vars
    +        _vars = _vars.copy()  # these will get clobbered in called functions
    +        nonlocal sig
    +        sig = copy.deepcopy(sig)  # just in case
    +        context = Context(
    +            vars=_vars,
    +            global_ctx=global_ctx,
    +            sigs=sigs,
    +            memory_allocator=memory_allocator,
    +            return_type=sig.return_type,
    +            constancy=Constancy.Constant
    +            if sig.mutability in ("view", "pure")
    +            else Constancy.Mutable,
    +            is_payable=sig.mutability == "payable",
    +            is_internal=sig.internal,
    +            sig=sig,
    +        )
    +
    +        if sig.internal:
    +            o = generate_lll_for_internal_function(code, sig, context)
    +        else:
    +            o = generate_lll_for_external_function(code, sig, context, check_nonpayable)
    +        return o, context
    +
    +    _, context = _run_pass(memory_allocator=None)
    +
    +    allocate_start = context.max_callee_frame_size
    +    allocate_start += MemoryPositions.RESERVED_MEMORY
    +
    +    o, context = _run_pass(memory_allocator=MemoryAllocator(allocate_start))
    +
    +    frame_size = context.memory_allocator.size_of_mem - MemoryPositions.RESERVED_MEMORY
    +
    +    if not sig.internal:
    +        # frame_size of external function includes all private functions called
    +        o.total_gas = o.gas + calc_mem_gas(frame_size)
    +    else:
    +        # frame size for internal function does not need to be adjusted
    +        # since it is already accounted for by the caller
    +        o.total_gas = o.gas
    +
    +    o.context = context
    +    o.func_name = sig.name
    +    return o, allocate_start, frame_size
    
  • vyper/old_codegen/function_definitions/external_function.py+219 0 added
    @@ -0,0 +1,219 @@
    +from typing import Any, List
    +
    +import vyper.utils as util
    +from vyper.ast.signatures.function_signature import (
    +    FunctionSignature,
    +    VariableRecord,
    +)
    +from vyper.exceptions import CompilerPanic
    +from vyper.old_codegen.context import Context
    +from vyper.old_codegen.expr import Expr
    +from vyper.old_codegen.function_definitions.utils import get_nonreentrant_lock
    +from vyper.old_codegen.lll_node import Encoding, LLLnode
    +from vyper.old_codegen.parser_utils import get_element_ptr, getpos, make_setter
    +from vyper.old_codegen.stmt import parse_body
    +from vyper.old_codegen.types.types import (
    +    BaseType,
    +    ByteArrayLike,
    +    ListType,
    +    TupleLike,
    +    TupleType,
    +)
    +
    +
    +def _should_decode(typ):
    +    # either a basetype which needs to be clamped
    +    # or a complex type which contains something that
    +    # needs to be clamped.
    +    if isinstance(typ, BaseType):
    +        return typ.typ not in ("int256", "uint256", "bytes32")
    +    if isinstance(typ, ByteArrayLike):
    +        return True
    +    if isinstance(typ, ListType):
    +        return _should_decode(typ.subtype)
    +    if isinstance(typ, TupleLike):
    +        return any(_should_decode(t) for t in typ.tuple_members())
    +    raise CompilerPanic(f"_should_decode({typ})")
    +
    +
    +# register function args with the local calling context.
    +# also allocate the ones that live in memory (i.e. kwargs)
    +def _register_function_args(context: Context, sig: FunctionSignature) -> List[LLLnode]:
    +    pos = None
    +
    +    ret = []
    +
    +    # the type of the calldata
    +    base_args_t = TupleType([arg.typ for arg in sig.base_args])
    +
    +    # tuple with the abi_encoded args
    +    if sig.is_init_func:
    +        base_args_ofst = LLLnode(
    +            "~codelen", location="code", typ=base_args_t, encoding=Encoding.ABI
    +        )
    +    else:
    +        base_args_ofst = LLLnode(4, location="calldata", typ=base_args_t, encoding=Encoding.ABI)
    +
    +    for i, arg in enumerate(sig.base_args):
    +
    +        arg_lll = get_element_ptr(base_args_ofst, i, pos=pos)
    +
    +        if _should_decode(arg.typ):
    +            # allocate a memory slot for it and copy
    +            p = context.new_variable(arg.name, arg.typ, is_mutable=False)
    +            dst = LLLnode(p, typ=arg.typ, location="memory")
    +            ret.append(make_setter(dst, arg_lll, "memory", pos=pos))
    +        else:
    +            # leave it in place
    +            context.vars[arg.name] = VariableRecord(
    +                name=arg.name,
    +                pos=arg_lll,
    +                typ=arg.typ,
    +                mutable=False,
    +                location=arg_lll.location,
    +                encoding=Encoding.ABI,
    +            )
    +
    +    return ret
    +
    +
    +def _annotated_method_id(abi_sig):
    +    method_id = util.abi_method_id(abi_sig)
    +    annotation = f"{hex(method_id)}: {abi_sig}"
    +    return LLLnode(method_id, annotation=annotation)
    +
    +
    +def _generate_kwarg_handlers(context: Context, sig: FunctionSignature, pos: Any) -> List[Any]:
    +    # generate kwarg handlers.
    +    # since they might come in thru calldata or be default,
    +    # allocate them in memory and then fill it in based on calldata or default,
    +    # depending on the signature
    +    # a kwarg handler looks like
    +    # (if (eq _method_id <method_id>)
    +    #    copy calldata args to memory
    +    #    write default args to memory
    +    #    goto external_function_common_lll
    +
    +    def handler_for(calldata_kwargs, default_kwargs):
    +        calldata_args = sig.base_args + calldata_kwargs
    +        # create a fake type so that get_element_ptr works
    +        calldata_args_t = TupleType(list(arg.typ for arg in calldata_args))
    +
    +        abi_sig = sig.abi_signature_for_kwargs(calldata_kwargs)
    +        method_id = _annotated_method_id(abi_sig)
    +
    +        calldata_kwargs_ofst = LLLnode(
    +            4, location="calldata", typ=calldata_args_t, encoding=Encoding.ABI
    +        )
    +
    +        # a sequence of statements to strictify kwargs into memory
    +        ret = ["seq"]
    +
    +        # TODO optimize make_setter by using
    +        # TupleType(list(arg.typ for arg in calldata_kwargs + default_kwargs))
    +        # (must ensure memory area is contiguous)
    +
    +        lhs_location = "memory"
    +        n_base_args = len(sig.base_args)
    +
    +        for i, arg_meta in enumerate(calldata_kwargs):
    +            k = n_base_args + i
    +
    +            dst = context.lookup_var(arg_meta.name).pos
    +
    +            lhs = LLLnode(dst, location="memory", typ=arg_meta.typ)
    +            rhs = get_element_ptr(calldata_kwargs_ofst, k, pos=None, array_bounds_check=False)
    +            ret.append(make_setter(lhs, rhs, lhs_location, pos))
    +
    +        for x in default_kwargs:
    +            dst = context.lookup_var(x.name).pos
    +            lhs = LLLnode(dst, location="memory", typ=x.typ)
    +            kw_ast_val = sig.default_values[x.name]  # e.g. `3` in x: int = 3
    +            rhs = Expr(kw_ast_val, context).lll_node
    +            ret.append(make_setter(lhs, rhs, lhs_location, pos))
    +
    +        ret.append(["goto", sig.external_function_base_entry_label])
    +
    +        ret = ["if", ["eq", "_calldata_method_id", method_id], ret]
    +        return ret
    +
    +    ret = ["seq"]
    +
    +    keyword_args = sig.default_args
    +
    +    # allocate variable slots in memory
    +    for arg in keyword_args:
    +        context.new_variable(arg.name, arg.typ, is_mutable=False)
    +
    +    for i, _ in enumerate(keyword_args):
    +        calldata_kwargs = keyword_args[:i]
    +        default_kwargs = keyword_args[i:]
    +
    +        ret.append(handler_for(calldata_kwargs, default_kwargs))
    +
    +    ret.append(handler_for(keyword_args, []))
    +
    +    return ret
    +
    +
    +# TODO it would be nice if this returned a data structure which were
    +# amenable to generating a jump table instead of the linear search for
    +# method_id we have now.
    +def generate_lll_for_external_function(code, sig, context, check_nonpayable):
    +    # TODO type hints:
    +    # def generate_lll_for_external_function(
    +    #    code: vy_ast.FunctionDef, sig: FunctionSignature, context: Context, check_nonpayable: bool,
    +    # ) -> LLLnode:
    +    """Return the LLL for an external function. Includes code to inspect the method_id,
    +       enter the function (nonpayable and reentrancy checks), handle kwargs and exit
    +       the function (clean up reentrancy storage variables)
    +    """
    +    func_type = code._metadata["type"]
    +    pos = getpos(code)
    +
    +    nonreentrant_pre, nonreentrant_post = get_nonreentrant_lock(func_type)
    +
    +    # generate handlers for base args and register the variable records
    +    handle_base_args = _register_function_args(context, sig)
    +
    +    # generate handlers for kwargs and register the variable records
    +    kwarg_handlers = _generate_kwarg_handlers(context, sig, pos)
    +
    +    # once optional args have been handled,
    +    # generate the main body of the function
    +    entrance = [["label", sig.external_function_base_entry_label]]
    +
    +    entrance += handle_base_args
    +
    +    if check_nonpayable and sig.mutability != "payable":
    +        # if the contract contains payable functions, but this is not one of them
    +        # add an assertion that the value of the call is zero
    +        entrance += [["assert", ["iszero", "callvalue"]]]
    +
    +    entrance += nonreentrant_pre
    +
    +    body = [parse_body(c, context) for c in code.body]
    +
    +    exit = [["label", sig.exit_sequence_label]] + nonreentrant_post
    +    if sig.is_init_func:
    +        pass  # init func has special exit sequence generated by parser.py
    +    elif context.return_type is None:
    +        exit += [["stop"]]
    +    else:
    +        # ret_ofst and ret_len stack items passed by function body; consume using 'pass'
    +        exit += [["return", "pass", "pass"]]
    +
    +    # the lll which comprises the main body of the function,
    +    # besides any kwarg handling
    +    func_common_lll = ["seq"] + entrance + body + exit
    +
    +    if sig.is_default_func or sig.is_init_func:
    +        # default and init funcs have special entries generated by parser.py
    +        ret = func_common_lll
    +    else:
    +        ret = kwarg_handlers
    +        # sneak the base code into the kwarg handler
    +        # TODO rethink this / make it clearer
    +        ret[-1][-1].append(func_common_lll)
    +
    +    return LLLnode.from_list(ret, pos=getpos(code))
    
  • vyper/old_codegen/function_definitions/__init__.py+2 2 modified
    @@ -1,5 +1,5 @@
    -from .parse_function import (  # noqa
    +from .common import (  # noqa
    +    generate_lll_for_function,
         is_default_func,
         is_initializer,
    -    parse_function,
     )
    
  • vyper/old_codegen/function_definitions/internal_function.py+64 0 added
    @@ -0,0 +1,64 @@
    +from vyper import ast as vy_ast
    +from vyper.ast.signatures import FunctionSignature
    +from vyper.old_codegen.context import Context
    +from vyper.old_codegen.function_definitions.utils import get_nonreentrant_lock
    +from vyper.old_codegen.lll_node import LLLnode
    +from vyper.old_codegen.parser_utils import getpos
    +from vyper.old_codegen.stmt import parse_body
    +
    +
    +def generate_lll_for_internal_function(
    +    code: vy_ast.FunctionDef, sig: FunctionSignature, context: Context
    +) -> LLLnode:
    +    """
    +    Parse a internal function (FuncDef), and produce full function body.
    +
    +    :param sig: the FuntionSignature
    +    :param code: ast of function
    +    :param context: current calling context
    +    :return: function body in LLL
    +    """
    +
    +    # The calling convention is:
    +    #   Caller fills in argument buffer
    +    #   Caller provides return address, return buffer on the stack
    +    #   Callee runs its code, fills in return buffer provided by caller
    +    #   Callee jumps back to caller
    +
    +    # The reason caller fills argument buffer is so there is less
    +    # complication with passing args on the stack; the caller is better
    +    # suited to optimize the copy operation. Also it avoids the callee
    +    # having to handle default args; that is easier left to the caller
    +    # as well. Meanwhile, the reason the callee fills the return buffer
    +    # is first, similarly, the callee is more suited to optimize the copy
    +    # operation. Second, it allows the caller to allocate the return
    +    # buffer in a way which reduces the number of copies. Third, it
    +    # reduces the potential for bugs since it forces the caller to have
    +    # the return data copied into a preallocated location. Otherwise, a
    +    # situation like the following is easy to bork:
    +    #   x: T[2] = [self.generate_T(), self.generate_T()]
    +
    +    func_type = code._metadata["type"]
    +
    +    # Get nonreentrant lock
    +
    +    for arg in sig.args:
    +        # allocate a variable for every arg, setting mutability
    +        # to False to comply with vyper semantics, function arguments are immutable
    +        context.new_variable(arg.name, arg.typ, is_mutable=False)
    +
    +    nonreentrant_pre, nonreentrant_post = get_nonreentrant_lock(func_type)
    +
    +    function_entry_label = sig.internal_function_label
    +    cleanup_label = sig.exit_sequence_label
    +
    +    # jump to the label which was passed in via stack
    +    stop_func = LLLnode.from_list(["jump", "pass"], annotation="jump to return address")
    +
    +    enter = [["label", function_entry_label]] + nonreentrant_pre
    +
    +    body = [parse_body(c, context) for c in code.body]
    +
    +    exit = [["label", cleanup_label]] + nonreentrant_post + [stop_func]
    +
    +    return LLLnode.from_list(["seq"] + enter + body + exit, typ=None, pos=getpos(code),)
    
  • vyper/old_codegen/function_definitions/parse_external_function.py+0 236 removed
    @@ -1,236 +0,0 @@
    -from typing import Any, List, Union
    -
    -from vyper import ast as vy_ast
    -from vyper.ast.signatures import sig_utils
    -from vyper.ast.signatures.function_signature import FunctionSignature
    -from vyper.old_codegen.arg_clamps import make_arg_clamper
    -from vyper.old_codegen.context import Context, VariableRecord
    -from vyper.old_codegen.expr import Expr
    -from vyper.old_codegen.function_definitions.utils import (
    -    get_default_names_to_set,
    -    get_nonreentrant_lock,
    -    get_sig_statements,
    -)
    -from vyper.old_codegen.lll_node import LLLnode
    -from vyper.old_codegen.parser_utils import getpos, make_setter
    -from vyper.old_codegen.stmt import parse_body
    -from vyper.old_codegen.types.types import ByteArrayLike, get_size_of_type
    -from vyper.utils import MemoryPositions
    -
    -
    -def get_external_arg_copier(
    -    total_size: int, memory_dest: int, offset: Union[int, List[Any]] = 4
    -) -> List[Any]:
    -    """
    -    Generate argument copier.
    -
    -    :param total_size: total memory size to copy
    -    :param memory_dest: base memory address to start from
    -    :param offset: starting offset, used for ByteArrays
    -    """
    -    copier = ["calldatacopy", memory_dest, offset, total_size]
    -    return copier
    -
    -
    -def parse_external_function(
    -    code: vy_ast.FunctionDef, sig: FunctionSignature, context: Context, check_nonpayable: bool,
    -) -> LLLnode:
    -    """
    -    Parse a external function (FuncDef), and produce full function body.
    -
    -    :param sig: the FuntionSignature
    -    :param code: ast of function
    -    :param check_nonpayable: if True, include a check that `msg.value == 0`
    -                             at the beginning of the function
    -    :return: full sig compare & function body
    -    """
    -
    -    func_type = code._metadata["type"]
    -
    -    # Get nonreentrant lock
    -    nonreentrant_pre, nonreentrant_post = get_nonreentrant_lock(func_type)
    -
    -    clampers = []
    -
    -    # Generate copiers
    -    copier: List[Any] = ["pass"]
    -    if not len(sig.base_args):
    -        copier = ["pass"]
    -    elif sig.name == "__init__":
    -        copier = ["codecopy", MemoryPositions.RESERVED_MEMORY, "~codelen", sig.base_copy_size]
    -        context.memory_allocator.expand_memory(sig.max_copy_size)
    -    clampers.append(copier)
    -
    -    if check_nonpayable and sig.mutability != "payable":
    -        # if the contract contains payable functions, but this is not one of them
    -        # add an assertion that the value of the call is zero
    -        clampers.append(["assert", ["iszero", "callvalue"]])
    -
    -    # Fill variable positions
    -    default_args_start_pos = len(sig.base_args)
    -    for i, arg in enumerate(sig.args):
    -        if i < len(sig.base_args):
    -            clampers.append(
    -                make_arg_clamper(
    -                    arg.pos,
    -                    context.memory_allocator.get_next_memory_position(),
    -                    arg.typ,
    -                    sig.name == "__init__",
    -                )
    -            )
    -        if isinstance(arg.typ, ByteArrayLike):
    -            mem_pos = context.memory_allocator.expand_memory(32 * get_size_of_type(arg.typ))
    -            context.vars[arg.name] = VariableRecord(arg.name, mem_pos, arg.typ, False)
    -        else:
    -            if sig.name == "__init__":
    -                context.vars[arg.name] = VariableRecord(
    -                    arg.name, MemoryPositions.RESERVED_MEMORY + arg.pos, arg.typ, False,
    -                )
    -            elif i >= default_args_start_pos:  # default args need to be allocated in memory.
    -                type_size = get_size_of_type(arg.typ) * 32
    -                default_arg_pos = context.memory_allocator.expand_memory(type_size)
    -                context.vars[arg.name] = VariableRecord(
    -                    name=arg.name, pos=default_arg_pos, typ=arg.typ, mutable=False,
    -                )
    -            else:
    -                context.vars[arg.name] = VariableRecord(
    -                    name=arg.name, pos=4 + arg.pos, typ=arg.typ, mutable=False, location="calldata"
    -                )
    -
    -    # Create "clampers" (input well-formedness checkers)
    -    # Return function body
    -    if sig.name == "__init__":
    -        o = LLLnode.from_list(
    -            ["seq"] + clampers + [parse_body(code.body, context)],  # type: ignore
    -            pos=getpos(code),
    -        )
    -    # Is default function.
    -    elif sig.is_default_func():
    -        o = LLLnode.from_list(
    -            ["seq"] + clampers + [parse_body(code.body, context)] + [["stop"]],  # type: ignore
    -            pos=getpos(code),
    -        )
    -    # Is a normal function.
    -    else:
    -        # Function with default parameters.
    -        if sig.total_default_args > 0:
    -            function_routine = f"{sig.name}_{sig.method_id}"
    -            default_sigs = sig_utils.generate_default_arg_sigs(
    -                code, context.sigs, context.global_ctx
    -            )
    -            sig_chain: List[Any] = ["seq"]
    -
    -            for default_sig in default_sigs:
    -                sig_compare, _ = get_sig_statements(default_sig, getpos(code))
    -
    -                # Populate unset default variables
    -                set_defaults = []
    -                for arg_name in get_default_names_to_set(sig, default_sig):
    -                    value = Expr(sig.default_values[arg_name], context).lll_node
    -                    var = context.vars[arg_name]
    -                    left = LLLnode.from_list(
    -                        var.pos,
    -                        typ=var.typ,
    -                        location="memory",
    -                        pos=getpos(code),
    -                        mutable=var.mutable,
    -                    )
    -                    set_defaults.append(make_setter(left, value, "memory", pos=getpos(code)))
    -
    -                current_sig_arg_names = {x.name for x in default_sig.args}
    -                base_arg_names = {arg.name for arg in sig.base_args}
    -                copier_arg_count = len(default_sig.args) - len(sig.base_args)
    -                copier_arg_names = list(current_sig_arg_names - base_arg_names)
    -
    -                # Order copier_arg_names, this is very important.
    -                copier_arg_names = [x.name for x in default_sig.args if x.name in copier_arg_names]
    -
    -                # Variables to be populated from calldata/stack.
    -                default_copiers: List[Any] = []
    -                if copier_arg_count > 0:
    -                    # Get map of variables in calldata, with thier offsets
    -                    offset = 4
    -                    calldata_offset_map = {}
    -                    for arg in default_sig.args:
    -                        calldata_offset_map[arg.name] = offset
    -                        offset += (
    -                            32
    -                            if isinstance(arg.typ, ByteArrayLike)
    -                            else get_size_of_type(arg.typ) * 32
    -                        )
    -
    -                    # Copy default parameters from calldata.
    -                    for arg_name in copier_arg_names:
    -                        var = context.vars[arg_name]
    -                        calldata_offset = calldata_offset_map[arg_name]
    -
    -                        # Add clampers.
    -                        default_copiers.append(
    -                            make_arg_clamper(calldata_offset - 4, var.pos, var.typ,)
    -                        )
    -                        # Add copying code.
    -                        _offset: Union[int, List[Any]] = calldata_offset
    -                        if isinstance(var.typ, ByteArrayLike):
    -                            _offset = ["add", 4, ["calldataload", calldata_offset]]
    -                        default_copiers.append(
    -                            get_external_arg_copier(
    -                                memory_dest=var.pos, total_size=var.size * 32, offset=_offset,
    -                            )
    -                        )
    -
    -                    default_copiers.append(0)  # for over arching seq, POP
    -
    -                sig_chain.append(
    -                    [
    -                        "if",
    -                        sig_compare,
    -                        [
    -                            "seq",
    -                            ["seq"] + set_defaults if set_defaults else ["pass"],
    -                            ["seq_unchecked"] + default_copiers if default_copiers else ["pass"],
    -                            ["goto", function_routine],
    -                        ],
    -                    ]
    -                )
    -
    -            # Function with default parameters.
    -            function_jump_label = f"{sig.name}_{sig.method_id}_skip"
    -            o = LLLnode.from_list(
    -                [
    -                    "seq",
    -                    sig_chain,
    -                    [
    -                        "seq",
    -                        ["goto", function_jump_label],
    -                        ["label", function_routine],
    -                        ["seq"]
    -                        + nonreentrant_pre
    -                        + clampers
    -                        + [parse_body(c, context) for c in code.body]
    -                        + nonreentrant_post
    -                        + [["stop"]],
    -                        ["label", function_jump_label],
    -                    ],
    -                ],
    -                typ=None,
    -                pos=getpos(code),
    -            )
    -
    -        else:
    -            # Function without default parameters.
    -            sig_compare, _ = get_sig_statements(sig, getpos(code))
    -            o = LLLnode.from_list(
    -                [
    -                    "if",
    -                    sig_compare,
    -                    ["seq"]
    -                    + nonreentrant_pre
    -                    + clampers
    -                    + [parse_body(c, context) for c in code.body]
    -                    + nonreentrant_post
    -                    + [["stop"]],
    -                ],
    -                typ=None,
    -                pos=getpos(code),
    -            )
    -    return o
    
  • vyper/old_codegen/function_definitions/parse_function.py+0 67 removed
    @@ -1,67 +0,0 @@
    -# can't use from [module] import [object] because it breaks mocks in testing
    -from vyper.ast.signatures import FunctionSignature
    -from vyper.old_codegen import context as ctx
    -from vyper.old_codegen.context import Constancy
    -# NOTE black/isort conflict >>
    -from vyper.old_codegen.function_definitions.parse_external_function import (
    -    parse_external_function,
    -)
    -# NOTE black/isort conflict >>
    -from vyper.old_codegen.function_definitions.parse_internal_function import (
    -    parse_internal_function,
    -)
    -from vyper.old_codegen.memory_allocator import MemoryAllocator
    -from vyper.utils import calc_mem_gas
    -
    -
    -# Is a function the initializer?
    -def is_initializer(code):
    -    return code.name == "__init__"
    -
    -
    -# Is a function the default function?
    -def is_default_func(code):
    -    return code.name == "__default__"
    -
    -
    -def parse_function(code, sigs, global_ctx, check_nonpayable, _vars=None):
    -    """
    -    Parses a function and produces LLL code for the function, includes:
    -        - Signature method if statement
    -        - Argument handling
    -        - Clamping and copying of arguments
    -        - Function body
    -    """
    -    if _vars is None:
    -        _vars = {}
    -    sig = FunctionSignature.from_definition(code, sigs=sigs, custom_structs=global_ctx._structs,)
    -
    -    # Validate return statements.
    -    sig.validate_return_statement_balance()
    -
    -    # Create a local (per function) context.
    -    memory_allocator = MemoryAllocator()
    -    context = ctx.Context(
    -        vars=_vars,
    -        global_ctx=global_ctx,
    -        sigs=sigs,
    -        memory_allocator=memory_allocator,
    -        return_type=sig.output_type,
    -        constancy=Constancy.Constant if sig.mutability in ("view", "pure") else Constancy.Mutable,
    -        is_payable=sig.mutability == "payable",
    -        is_internal=sig.internal,
    -        method_id=sig.method_id,
    -        sig=sig,
    -    )
    -
    -    if sig.internal:
    -        o = parse_internal_function(code=code, sig=sig, context=context,)
    -    else:
    -        o = parse_external_function(
    -            code=code, sig=sig, context=context, check_nonpayable=check_nonpayable
    -        )
    -
    -    o.context = context
    -    o.total_gas = o.gas + calc_mem_gas(o.context.memory_allocator.size_of_mem)
    -    o.func_name = sig.name
    -    return o
    
  • vyper/old_codegen/function_definitions/parse_internal_function.py+0 244 removed
    @@ -1,244 +0,0 @@
    -from typing import Any, List
    -
    -from vyper import ast as vy_ast
    -from vyper.ast.signatures import FunctionSignature, sig_utils
    -from vyper.ast.signatures.function_signature import VariableRecord
    -from vyper.old_codegen.context import Context
    -from vyper.old_codegen.expr import Expr
    -from vyper.old_codegen.function_definitions.utils import (
    -    get_default_names_to_set,
    -    get_nonreentrant_lock,
    -    get_sig_statements,
    -    make_unpacker,
    -)
    -from vyper.old_codegen.lll_node import LLLnode
    -from vyper.old_codegen.parser_utils import getpos, make_setter
    -from vyper.old_codegen.stmt import parse_body
    -from vyper.old_codegen.types.types import (
    -    BaseType,
    -    ByteArrayLike,
    -    get_size_of_type,
    -)
    -from vyper.utils import MemoryPositions
    -
    -
    -def get_internal_arg_copier(total_size: int, memory_dest: int) -> List[Any]:
    -    """
    -    Copy arguments.
    -    For internal functions, MSTORE arguments and callback pointer from the stack.
    -
    -    :param  total_size: total size to copy
    -    :param  memory_dest: base memory position to copy to
    -    :return: LLL list that copies total_size of memory
    -    """
    -
    -    copier: List[Any] = ["seq"]
    -    for pos in range(0, total_size, 32):
    -        copier.append(["mstore", memory_dest + pos, "pass"])
    -    return copier
    -
    -
    -def parse_internal_function(
    -    code: vy_ast.FunctionDef, sig: FunctionSignature, context: Context
    -) -> LLLnode:
    -    """
    -    Parse a internal function (FuncDef), and produce full function body.
    -
    -    :param sig: the FuntionSignature
    -    :param code: ast of function
    -    :return: full sig compare & function body
    -    """
    -
    -    func_type = code._metadata["type"]
    -
    -    # Get nonreentrant lock
    -    nonreentrant_pre, nonreentrant_post = get_nonreentrant_lock(func_type)
    -
    -    # Create callback_ptr, this stores a destination in the bytecode for a internal
    -    # function to jump to after a function has executed.
    -    clampers: List[LLLnode] = []
    -
    -    # Allocate variable space.
    -    context.memory_allocator.expand_memory(sig.max_copy_size)
    -
    -    _post_callback_ptr = f"{sig.name}_{sig.method_id}_post_callback_ptr"
    -    context.callback_ptr = context.new_internal_variable(typ=BaseType("uint256"))
    -    clampers.append(
    -        LLLnode.from_list(
    -            ["mstore", context.callback_ptr, "pass"], annotation="pop callback pointer",
    -        )
    -    )
    -    if sig.total_default_args > 0:
    -        clampers.append(LLLnode.from_list(["label", _post_callback_ptr]))
    -
    -    # internal functions without return types need to jump back to
    -    # the calling function, as there is no return statement to handle the
    -    # jump.
    -    if sig.output_type is None:
    -        stop_func = [["jump", ["mload", context.callback_ptr]]]
    -    else:
    -        stop_func = [["stop"]]
    -
    -    # Generate copiers
    -    if len(sig.base_args) == 0:
    -        copier = ["pass"]
    -        clampers.append(LLLnode.from_list(copier))
    -    elif sig.total_default_args == 0:
    -        copier = get_internal_arg_copier(
    -            total_size=sig.base_copy_size, memory_dest=MemoryPositions.RESERVED_MEMORY
    -        )
    -        clampers.append(LLLnode.from_list(copier))
    -
    -    # Fill variable positions
    -    for arg in sig.args:
    -        if isinstance(arg.typ, ByteArrayLike):
    -            mem_pos = context.memory_allocator.expand_memory(32 * get_size_of_type(arg.typ))
    -            context.vars[arg.name] = VariableRecord(arg.name, mem_pos, arg.typ, False)
    -        else:
    -            context.vars[arg.name] = VariableRecord(
    -                arg.name, MemoryPositions.RESERVED_MEMORY + arg.pos, arg.typ, False,
    -            )
    -
    -    # internal function copiers. No clamping for internal functions.
    -    dyn_variable_names = [a.name for a in sig.base_args if isinstance(a.typ, ByteArrayLike)]
    -    if dyn_variable_names:
    -        i_placeholder = context.new_internal_variable(typ=BaseType("uint256"))
    -        unpackers: List[Any] = []
    -        for idx, var_name in enumerate(dyn_variable_names):
    -            var = context.vars[var_name]
    -            ident = f"_load_args_{sig.method_id}_dynarg{idx}"
    -            o = make_unpacker(ident=ident, i_placeholder=i_placeholder, begin_pos=var.pos)
    -            unpackers.append(o)
    -
    -        if not unpackers:
    -            unpackers = ["pass"]
    -
    -        # 0 added to complete full overarching 'seq' statement, see internal_label.
    -        unpackers.append(0)
    -        clampers.append(
    -            LLLnode.from_list(
    -                ["seq_unchecked"] + unpackers,
    -                typ=None,
    -                annotation="dynamic unpacker",
    -                pos=getpos(code),
    -            )
    -        )
    -
    -    # Function has default arguments.
    -    if sig.total_default_args > 0:  # Function with default parameters.
    -
    -        default_sigs = sig_utils.generate_default_arg_sigs(code, context.sigs, context.global_ctx)
    -        sig_chain: List[Any] = ["seq"]
    -
    -        for default_sig in default_sigs:
    -            sig_compare, internal_label = get_sig_statements(default_sig, getpos(code))
    -
    -            # Populate unset default variables
    -            set_defaults = []
    -            for arg_name in get_default_names_to_set(sig, default_sig):
    -                value = Expr(sig.default_values[arg_name], context).lll_node
    -                var = context.vars[arg_name]
    -                left = LLLnode.from_list(
    -                    var.pos, typ=var.typ, location="memory", pos=getpos(code), mutable=var.mutable
    -                )
    -                set_defaults.append(make_setter(left, value, "memory", pos=getpos(code)))
    -            current_sig_arg_names = [x.name for x in default_sig.args]
    -
    -            # Load all variables in default section, if internal,
    -            # because the stack is a linear pipe.
    -            copier_arg_count = len(default_sig.args)
    -            copier_arg_names = current_sig_arg_names
    -
    -            # Order copier_arg_names, this is very important.
    -            copier_arg_names = [x.name for x in default_sig.args if x.name in copier_arg_names]
    -
    -            # Variables to be populated from calldata/stack.
    -            default_copiers: List[Any] = []
    -            if copier_arg_count > 0:
    -                # Get map of variables in calldata, with thier offsets
    -                offset = 4
    -                calldata_offset_map = {}
    -                for arg in default_sig.args:
    -                    calldata_offset_map[arg.name] = offset
    -                    offset += (
    -                        32 if isinstance(arg.typ, ByteArrayLike) else get_size_of_type(arg.typ) * 32
    -                    )
    -
    -                # Copy set default parameters from calldata
    -                dynamics = []
    -                for arg_name in copier_arg_names:
    -                    var = context.vars[arg_name]
    -                    if isinstance(var.typ, ByteArrayLike):
    -                        _size = 32
    -                        dynamics.append(var.pos)
    -                    else:
    -                        _size = var.size * 32
    -                    default_copiers.append(
    -                        get_internal_arg_copier(memory_dest=var.pos, total_size=_size,)
    -                    )
    -
    -                # Unpack byte array if necessary.
    -                if dynamics:
    -                    i_placeholder = context.new_internal_variable(typ=BaseType("uint256"))
    -                    for idx, var_pos in enumerate(dynamics):
    -                        ident = f"unpack_default_sig_dyn_{default_sig.method_id}_arg{idx}"
    -                        default_copiers.append(
    -                            make_unpacker(
    -                                ident=ident, i_placeholder=i_placeholder, begin_pos=var_pos,
    -                            )
    -                        )
    -                default_copiers.append(0)  # for over arching seq, POP
    -
    -            sig_chain.append(
    -                [
    -                    "if",
    -                    sig_compare,
    -                    [
    -                        "seq",
    -                        internal_label,
    -                        LLLnode.from_list(
    -                            ["mstore", context.callback_ptr, "pass"],
    -                            annotation="pop callback pointer",
    -                            pos=getpos(code),
    -                        ),
    -                        ["seq"] + set_defaults if set_defaults else ["pass"],
    -                        ["seq_unchecked"] + default_copiers if default_copiers else ["pass"],
    -                        ["goto", _post_callback_ptr],
    -                    ],
    -                ]
    -            )
    -
    -        # With internal functions all variable loading occurs in the default
    -        # function sub routine.
    -        _clampers = [["label", _post_callback_ptr]]
    -
    -        # Function with default parameters.
    -        return LLLnode.from_list(
    -            [
    -                "seq",
    -                sig_chain,
    -                ["seq"]
    -                + nonreentrant_pre
    -                + _clampers
    -                + [parse_body(c, context) for c in code.body]
    -                + nonreentrant_post
    -                + stop_func,
    -            ],
    -            typ=None,
    -            pos=getpos(code),
    -        )
    -
    -    else:
    -        # Function without default parameters.
    -        sig_compare, internal_label = get_sig_statements(sig, getpos(code))
    -        return LLLnode.from_list(
    -            ["seq"]
    -            + [internal_label]
    -            + nonreentrant_pre
    -            + clampers
    -            + [parse_body(c, context) for c in code.body]
    -            + nonreentrant_post
    -            + stop_func,
    -            typ=None,
    -            pos=getpos(code),
    -        )
    
  • vyper/old_codegen/function_definitions/utils.py+6 59 modified
    @@ -1,61 +1,8 @@
    -from vyper.old_codegen.lll_node import LLLnode
    -
    -
    -def get_sig_statements(sig, pos):
    -    method_id_node = LLLnode.from_list(sig.method_id, pos=pos, annotation=f"{sig.sig}")
    -
    -    if sig.internal:
    -        sig_compare = 0
    -        private_label = LLLnode.from_list(
    -            ["label", f"priv_{sig.method_id}"], pos=pos, annotation=f"{sig.sig}"
    -        )
    -    else:
    -        sig_compare = ["eq", "_func_sig", method_id_node]
    -        private_label = ["pass"]
    -
    -    return sig_compare, private_label
    -
    -
    -def make_unpacker(ident, i_placeholder, begin_pos):
    -    start_label = "dyn_unpack_start_" + ident
    -    end_label = "dyn_unpack_end_" + ident
    -    return [
    -        "seq_unchecked",
    -        ["mstore", begin_pos, "pass"],  # get len
    -        ["mstore", i_placeholder, 0],
    -        ["label", start_label],
    -        [  # break
    -            "if",
    -            ["ge", ["mload", i_placeholder], ["ceil32", ["mload", begin_pos]]],
    -            ["goto", end_label],
    -        ],
    -        [  # pop into correct memory slot.
    -            "mstore",
    -            ["add", ["add", begin_pos, 32], ["mload", i_placeholder]],
    -            "pass",
    -        ],
    -        ["mstore", i_placeholder, ["add", 32, ["mload", i_placeholder]]],  # increment i
    -        ["goto", start_label],
    -        ["label", end_label],
    -    ]
    -
    -
     def get_nonreentrant_lock(func_type):
    -    nonreentrant_pre = [["pass"]]
    -    nonreentrant_post = [["pass"]]
    -    if func_type.nonreentrant:
    -        nkey = func_type.reentrancy_key_position.position
    -        nonreentrant_pre = [["seq", ["assert", ["iszero", ["sload", nkey]]], ["sstore", nkey, 1]]]
    -        nonreentrant_post = [["sstore", nkey, 0]]
    -    return nonreentrant_pre, nonreentrant_post
    +    if not func_type.nonreentrant:
    +        return ["pass"], ["pass"]
     
    -
    -def get_default_names_to_set(primary_sig, default_sig):
    -    """
    -    Get names for default parameters that require a default value to be assigned.
    -    """
    -
    -    current_sig_arg_names = [x.name for x in default_sig.args]
    -    for arg in primary_sig.default_args:
    -        if arg.arg not in current_sig_arg_names:
    -            yield arg.arg
    +    nkey = func_type.reentrancy_key_position.position
    +    nonreentrant_pre = [["seq", ["assert", ["iszero", ["sload", nkey]]], ["sstore", nkey, 1]]]
    +    nonreentrant_post = [["sstore", nkey, 0]]
    +    return nonreentrant_pre, nonreentrant_post
    
  • vyper/old_codegen/global_context.py+1 1 modified
    @@ -219,5 +219,5 @@ def add_globals_and_events(self, item):
             else:
                 raise InvalidType("Invalid global type specified", item)
     
    -    def parse_type(self, ast_node, location):
    +    def parse_type(self, ast_node, location=None):
             return parse_type(ast_node, location, sigs=self._contracts, custom_structs=self._structs,)
    
  • vyper/old_codegen/keccak256_helper.py+1 0 modified
    @@ -19,6 +19,7 @@ def _gas_bound(num_words):
         return SHA3_BASE + num_words * SHA3_PER_WORD
     
     
    +# TODO kwargs is dead argument
     def keccak256_helper(expr, lll_args, kwargs, context):
         if len(lll_args) != 1:
             # NOTE this may be checked at a higher level, but just be safe
    
  • vyper/old_codegen/lll_node.py+55 6 modified
    @@ -1,11 +1,12 @@
     import re
    +from enum import Enum, auto
     from typing import Any, List, Optional, Tuple, Union
     
     from vyper.compiler.settings import VYPER_COLOR_OUTPUT
     from vyper.evm.opcodes import get_comb_opcodes
     from vyper.exceptions import CompilerPanic
     from vyper.old_codegen.types import BaseType, NodeType, ceil32
    -from vyper.utils import VALID_LLL_MACROS
    +from vyper.utils import VALID_LLL_MACROS, cached_property
     
     # Set default string representation for ints in LLL output.
     AS_HEX_DEFAULT = False
    @@ -35,6 +36,21 @@ def __repr__(self) -> str:
         __mul__ = __add__
     
     
    +def push_label_to_stack(labelname: str) -> str:
    +    #  items prefixed with `_sym_` are ignored until asm phase
    +    return "_sym_" + labelname
    +
    +
    +class Encoding(Enum):
    +    # vyper encoding, default for memory variables
    +    VYPER = auto()
    +    # abi encoded, default for args/return values from external funcs
    +    ABI = auto()
    +    # abi encoded, same as ABI but no clamps for bytestrings
    +    JSON_ABI = auto()
    +    # future: packed
    +
    +
     # Data structure for LLL parse tree
     class LLLnode:
         repr_show_gas = False
    @@ -47,26 +63,29 @@ def __init__(
             self,
             value: Union[str, int],
             args: List["LLLnode"] = None,
    -        typ: "BaseType" = None,
    +        typ: NodeType = None,
             location: str = None,
             pos: Optional[Tuple[int, int]] = None,
             annotation: Optional[str] = None,
             mutable: bool = True,
             add_gas_estimate: int = 0,
             valency: Optional[int] = None,
    +        encoding: Encoding = Encoding.VYPER,
         ):
             if args is None:
                 args = []
     
             self.value = value
             self.args = args
    +        # TODO remove this sanity check once mypy is more thorough
    +        assert isinstance(typ, NodeType) or typ is None, repr(typ)
             self.typ = typ
    -        assert isinstance(self.typ, NodeType) or self.typ is None, repr(self.typ)
             self.location = location
             self.pos = pos
             self.annotation = annotation
             self.mutable = mutable
             self.add_gas_estimate = add_gas_estimate
    +        self.encoding = encoding
             self.as_hex = AS_HEX_DEFAULT
     
             # Optional annotation properties for gas estimation
    @@ -136,9 +155,11 @@ def __init__(
                 # With statements: with <var> <initial> <statement>
                 elif self.value == "with":
                     if len(self.args) != 3:
    -                    raise CompilerPanic("With statement must have 3 arguments")
    +                    raise CompilerPanic(f"With statement must have 3 arguments: {self}")
                     if len(self.args[0].args) or not isinstance(self.args[0].value, str):
    -                    raise CompilerPanic("First argument to with statement must be a variable")
    +                    raise CompilerPanic(
    +                        f"First argument to with statement must be a variable: {self}"
    +                    )
                     if not self.args[1].valency and self.args[1].value != "pass":
                         raise CompilerPanic(
                             (
    @@ -203,6 +224,16 @@ def __init__(
                 elif self.value == "seq":
                     self.valency = self.args[-1].valency if self.args else 0
                     self.gas = sum([arg.gas for arg in self.args]) + 30
    +
    +            # GOTO is a jump with args
    +            # e.g. (goto my_label x y z) will push x y and z onto the stack,
    +            # then JUMP to my_label.
    +            elif self.value == "goto":
    +                for arg in self.args:
    +                    if not arg.valency and arg.value != "pass":
    +                        raise CompilerPanic(f"zerovalent argument to goto {self}")
    +                self.valency = 0
    +                self.gas = sum([arg.gas for arg in self.args])
                 # Multi statements: multi <expr> <expr> ...
                 elif self.value == "multi":
                     for arg in self.args:
    @@ -238,6 +269,17 @@ def __init__(
     
             self.gas += self.add_gas_estimate
     
    +    # the LLL should be cached.
    +    @property
    +    def is_complex_lll(self):
    +        return isinstance(self.value, str) and (
    +            self.value.lower() in VALID_LLL_MACROS or self.value.upper() in get_comb_opcodes()
    +        )
    +
    +    @cached_property
    +    def contains_self_call(self):
    +        return getattr(self, "is_self_call", False) or any(x.contains_self_call for x in self.args)
    +
         def __getitem__(self, i):
             return self.to_list()[i]
     
    @@ -328,13 +370,14 @@ def __repr__(self):
         def from_list(
             cls,
             obj: Any,
    -        typ: "BaseType" = None,
    +        typ: NodeType = None,
             location: str = None,
             pos: Tuple[int, int] = None,
             annotation: Optional[str] = None,
             mutable: bool = True,
             add_gas_estimate: int = 0,
             valency: Optional[int] = None,
    +        encoding: Encoding = Encoding.VYPER,
         ) -> "LLLnode":
             if isinstance(typ, str):
                 typ = BaseType(typ)
    @@ -348,6 +391,9 @@ def from_list(
                     obj.pos = pos
                 if obj.location is None:
                     obj.location = location
    +            if obj.encoding is None:
    +                obj.encoding = encoding
    +
                 return obj
             elif not isinstance(obj, list):
                 return cls(
    @@ -359,6 +405,8 @@ def from_list(
                     annotation=annotation,
                     mutable=mutable,
                     add_gas_estimate=add_gas_estimate,
    +                valency=valency,
    +                encoding=encoding,
                 )
             else:
                 return cls(
    @@ -371,4 +419,5 @@ def from_list(
                     mutable=mutable,
                     add_gas_estimate=add_gas_estimate,
                     valency=valency,
    +                encoding=encoding,
                 )
    
  • vyper/old_codegen/parser.py+66 46 modified
    @@ -1,17 +1,16 @@
    -from typing import Any, List, Optional, Tuple
    +from typing import Any, List, Optional, Tuple, Union
     
     from vyper import ast as vy_ast
    -from vyper.ast.signatures import sig_utils
     from vyper.ast.signatures.function_signature import FunctionSignature
     from vyper.exceptions import (
         EventDeclarationException,
         FunctionDeclarationException,
         StructureException,
     )
     from vyper.old_codegen.function_definitions import (
    +    generate_lll_for_function,
         is_default_func,
         is_initializer,
    -    parse_function,
     )
     from vyper.old_codegen.global_context import GlobalContext
     from vyper.old_codegen.lll_node import LLLnode
    @@ -29,7 +28,7 @@
         # check that calldatasize is at least 4, otherwise
         # calldataload will load zeros (cf. yellow paper).
         ["if", ["lt", "calldatasize", 4], ["goto", "fallback"]],
    -    ["mstore", 28, ["calldataload", 0]],
    +    ["calldatacopy", 28, 0, 4],
     ]
     # Store limit constants at fixed addresses in memory.
     LIMIT_MEMORY_SET: List[Any] = [
    @@ -47,6 +46,7 @@ def init_func_init_lll():
     
     def parse_external_interfaces(external_interfaces, global_ctx):
         for _interfacename in global_ctx._contracts:
    +        # TODO factor me into helper function
             _interface_defs = global_ctx._contracts[_interfacename]
             _defnames = [_def.name for _def in _interface_defs]
             interface = {}
    @@ -87,79 +87,99 @@ def parse_external_interfaces(external_interfaces, global_ctx):
         return external_interfaces
     
     
    -def parse_other_functions(o, otherfuncs, sigs, external_interfaces, global_ctx, default_function):
    +def parse_regular_functions(
    +    o, regular_functions, sigs, external_interfaces, global_ctx, default_function
    +):
         # check for payable/nonpayable external functions to optimize nonpayable assertions
         func_types = [i._metadata["type"] for i in global_ctx._defs]
         mutabilities = [i.mutability for i in func_types if i.visibility == FunctionVisibility.EXTERNAL]
    -    has_payable = next((True for i in mutabilities if i == StateMutability.PAYABLE), False)
    -    has_nonpayable = next((True for i in mutabilities if i != StateMutability.PAYABLE), False)
    +    has_payable = any(i == StateMutability.PAYABLE for i in mutabilities)
    +    has_nonpayable = any(i != StateMutability.PAYABLE for i in mutabilities)
    +
         is_default_payable = (
             default_function is not None
             and default_function._metadata["type"].mutability == StateMutability.PAYABLE
         )
    +
    +    # TODO streamline the nonpayable check logic
    +
         # when a contract has a payable default function and at least one nonpayable
         # external function, we must perform the nonpayable check on every function
         check_per_function = is_default_payable and has_nonpayable
     
         # generate LLL for regular functions
    -    payable_func_sub = ["seq"]
    -    external_func_sub = ["seq"]
    -    internal_func_sub = ["seq"]
    +    payable_funcs = []
    +    nonpayable_funcs = []
    +    internal_funcs = []
         add_gas = func_init_lll().gas
     
    -    for func_node in otherfuncs:
    +    for func_node in regular_functions:
             func_type = func_node._metadata["type"]
    -        func_lll = parse_function(
    +        func_lll, frame_start, frame_size = generate_lll_for_function(
                 func_node, {**{"self": sigs}, **external_interfaces}, global_ctx, check_per_function
             )
    +
             if func_type.visibility == FunctionVisibility.INTERNAL:
    -            internal_func_sub.append(func_lll)
    +            internal_funcs.append(func_lll)
    +
             elif func_type.mutability == StateMutability.PAYABLE:
    -            add_gas += 30
    -            payable_func_sub.append(func_lll)
    +            add_gas += 30  # CMC 20210910 why?
    +            payable_funcs.append(func_lll)
    +
             else:
    -            external_func_sub.append(func_lll)
    -            add_gas += 30
    +            add_gas += 30  # CMC 20210910 why?
    +            nonpayable_funcs.append(func_lll)
    +
             func_lll.total_gas += add_gas
    -        for sig in sig_utils.generate_default_arg_sigs(func_node, external_interfaces, global_ctx):
    -            sig.gas = func_lll.total_gas
    -            sigs[sig.sig] = sig
    +
    +        # update sigs with metadata gathered from compiling the function so that
    +        # we can handle calls to self
    +        # TODO we only need to do this for internal functions; external functions
    +        # cannot be called via `self`
    +        sig = FunctionSignature.from_definition(func_node, external_interfaces, global_ctx._structs)
    +        sig.gas = func_lll.total_gas
    +        sig.frame_start = frame_start
    +        sig.frame_size = frame_size
    +        sigs[sig.name] = sig
     
         # generate LLL for fallback function
         if default_function:
    -        fallback_lll = parse_function(
    +        fallback_lll, _frame_start, _frame_size = generate_lll_for_function(
                 default_function,
                 {**{"self": sigs}, **external_interfaces},
                 global_ctx,
                 # include a nonpayble check here if the contract only has a default function
    -            check_per_function or not otherfuncs,
    +            check_per_function or not regular_functions,
             )
         else:
             fallback_lll = LLLnode.from_list(["revert", 0, 0], typ=None, annotation="Default function")
     
         if check_per_function:
    -        external_seq = ["seq", payable_func_sub, external_func_sub]
    +        external_seq = ["seq"] + payable_funcs + nonpayable_funcs
    +
         else:
             # payable functions are placed prior to nonpayable functions
             # and seperated by a nonpayable assertion
             external_seq = ["seq"]
             if has_payable:
    -            external_seq.append(payable_func_sub)
    +            external_seq += payable_funcs
             if has_nonpayable:
    -            external_seq.extend([["assert", ["iszero", "callvalue"]], external_func_sub])
    +            external_seq.append(["assert", ["iszero", "callvalue"]])
    +            external_seq += nonpayable_funcs
     
         # bytecode is organized by: external functions, fallback fn, internal functions
         # this way we save gas and reduce bytecode by not jumping over internal functions
    -    main_seq = [
    +    runtime = [
             "seq",
             func_init_lll(),
    -        ["with", "_func_sig", ["mload", 0], external_seq],
    +        ["with", "_calldata_method_id", ["mload", 0], external_seq],
             ["seq_unchecked", ["label", "fallback"], fallback_lll],
    -        internal_func_sub,
         ]
    +    runtime.extend(internal_funcs)
     
    -    o.append(["return", 0, ["lll", main_seq, 0]])
    -    return o, main_seq
    +    # TODO CMC 20210911 why does the lll have a trailing 0
    +    o.append(["return", 0, ["lll", runtime, 0]])
    +    return o, runtime
     
     
     # Main python parse tree => LLL method
    @@ -179,40 +199,40 @@ def parse_tree_to_lll(global_ctx: GlobalContext) -> Tuple[LLLnode, LLLnode]:
                 {[name for name in _names_events if _names_events.count(name) > 1][0]}"""
             )
         # Initialization function
    -    initfunc = [_def for _def in global_ctx._defs if is_initializer(_def)]
    +    init_function = next((_def for _def in global_ctx._defs if is_initializer(_def)), None)
         # Default function
    -    defaultfunc = next((i for i in global_ctx._defs if is_default_func(i)), None)
    -    # Regular functions
    -    otherfuncs = [
    +    default_function = next((i for i in global_ctx._defs if is_default_func(i)), None)
    +
    +    regular_functions = [
             _def for _def in global_ctx._defs if not is_initializer(_def) and not is_default_func(_def)
         ]
     
         sigs: dict = {}
         external_interfaces: dict = {}
         # Create the main statement
    -    o = ["seq"]
    +    o: List[Union[str, LLLnode]] = ["seq"]
         if global_ctx._contracts or global_ctx._interfaces:
             external_interfaces = parse_external_interfaces(external_interfaces, global_ctx)
    -    # If there is an init func...
    -    if initfunc:
    +
    +    # TODO: fix for #2251 is to move this after parse_regular_functions
    +    if init_function:
             o.append(init_func_init_lll())
    -        o.append(
    -            parse_function(
    -                initfunc[0], {**{"self": sigs}, **external_interfaces}, global_ctx, False,
    -            )
    +        init_func_lll, _frame_start, _frame_size = generate_lll_for_function(
    +            init_function, {**{"self": sigs}, **external_interfaces}, global_ctx, False,
             )
    +        o.append(init_func_lll)
     
    -    # If there are regular functions...
    -    if otherfuncs or defaultfunc:
    -        o, runtime = parse_other_functions(
    -            o, otherfuncs, sigs, external_interfaces, global_ctx, defaultfunc,
    +    if regular_functions or default_function:
    +        o, runtime = parse_regular_functions(
    +            o, regular_functions, sigs, external_interfaces, global_ctx, default_function,
             )
         else:
             runtime = o.copy()
     
    -    return LLLnode.from_list(o, typ=None), LLLnode.from_list(runtime, typ=None)
    +    return LLLnode.from_list(o), LLLnode.from_list(runtime)
     
     
    +# TODO this function is dead code
     def parse_to_lll(
         source_code: str, runtime_only: bool = False, interface_codes: Optional[InterfaceImports] = None
     ) -> LLLnode:
    
  • vyper/old_codegen/parser_utils.py+342 242 modified
    @@ -1,6 +1,7 @@
     from decimal import Decimal, getcontext
     
     from vyper import ast as vy_ast
    +from vyper.evm.opcodes import version_check
     from vyper.exceptions import (
         CompilerPanic,
         InvalidLiteral,
    @@ -9,22 +10,26 @@
         TypeMismatch,
     )
     from vyper.old_codegen.arg_clamps import int128_clamp
    -from vyper.old_codegen.lll_node import LLLnode
    +from vyper.old_codegen.lll_node import Encoding, LLLnode
     from vyper.old_codegen.types import (
         BaseType,
         ByteArrayLike,
    -    ByteArrayType,
         ListType,
         MappingType,
         StructType,
         TupleLike,
         TupleType,
         ceil32,
         get_size_of_type,
    -    has_dynamic_data,
         is_base_type,
     )
    -from vyper.utils import GAS_IDENTITY, GAS_IDENTITYWORD, MemoryPositions
    +from vyper.utils import (
    +    GAS_CALLDATACOPY_WORD,
    +    GAS_CODECOPY_WORD,
    +    GAS_IDENTITY,
    +    GAS_IDENTITYWORD,
    +    MemoryPositions,
    +)
     
     getcontext().prec = 78  # MAX_UINT256 < 1e78
     
    @@ -33,7 +38,7 @@ def type_check_wrapper(fn):
         def _wrapped(*args, **kwargs):
             return_value = fn(*args, **kwargs)
             if return_value is None:
    -            raise TypeCheckFailure(f"{fn.__name__} did not return a value")
    +            raise TypeCheckFailure(f"{fn.__name__} {args} did not return a value")
             return return_value
     
         return _wrapped
    @@ -67,11 +72,18 @@ def _identity_gas_bound(num_bytes):
         return GAS_IDENTITY + GAS_IDENTITYWORD * (ceil32(num_bytes) // 32)
     
     
    +def _calldatacopy_gas_bound(num_bytes):
    +    return GAS_CALLDATACOPY_WORD * ceil32(num_bytes) // 32
    +
    +
    +def _codecopy_gas_bound(num_bytes):
    +    return GAS_CODECOPY_WORD * ceil32(num_bytes) // 32
    +
    +
     # Copy byte array word-for-word (including layout)
     def make_byte_array_copier(destination, source, pos=None):
         if not isinstance(source.typ, ByteArrayLike):
    -        btype = "byte array" if isinstance(destination.typ, ByteArrayType) else "string"
    -        raise TypeMismatch(f"Can only set a {btype} to another {btype}", pos)
    +        raise TypeMismatch(f"Cannot cast from {source.typ} to {destination.typ}", pos)
         if isinstance(source.typ, ByteArrayLike) and source.typ.maxlen > destination.typ.maxlen:
             raise TypeMismatch(
                 f"Cannot cast from greater max-length {source.typ.maxlen} to shorter "
    @@ -86,22 +98,24 @@ def make_byte_array_copier(destination, source, pos=None):
                 )
     
         # Special case: memory to memory
    -    if source.location == "memory" and destination.location == "memory":
    +    # TODO: this should be handled by make_byte_slice_copier.
    +    if destination.location == "memory" and source.location in ("memory", "code", "calldata"):
    +        if source.location == "memory":
    +            # TODO turn this into an LLL macro: memorycopy
    +            copy_op = ["assert", ["call", ["gas"], 4, 0, "src", "sz", destination, "sz"]]
    +            gas_bound = _identity_gas_bound(source.typ.maxlen)
    +        elif source.location == "calldata":
    +            copy_op = ["calldatacopy", destination, "src", "sz"]
    +            gas_bound = _calldatacopy_gas_bound(source.typ.maxlen)
    +        elif source.location == "code":
    +            copy_op = ["codecopy", destination, "src", "sz"]
    +            gas_bound = _codecopy_gas_bound(source.typ.maxlen)
    +        _sz_lll = ["add", 32, [load_op(source.location), "src"]]
             o = LLLnode.from_list(
    -            [
    -                "with",
    -                "_source",
    -                source,
    -                [
    -                    "with",
    -                    "_sz",
    -                    ["add", 32, ["mload", "_source"]],
    -                    ["assert", ["call", ["gas"], 4, 0, "_source", "_sz", destination, "_sz"]],
    -                ],
    -            ],  # noqa: E501
    +            ["with", "src", source, ["with", "sz", _sz_lll, copy_op]],
                 typ=None,
    -            add_gas_estimate=_identity_gas_bound(source.typ.maxlen),
    -            annotation="Memory copy",
    +            add_gas_estimate=gas_bound,
    +            annotation="copy bytestring to memory",
             )
             return o
     
    @@ -112,13 +126,13 @@ def make_byte_array_copier(destination, source, pos=None):
         # Get the length
         if source.value is None:
             length = 1
    -    elif source.location == "memory":
    -        length = ["add", ["mload", "_pos"], 32]
    +    elif source.location in ("memory", "code", "calldata"):
    +        length = ["add", [load_op(source.location), "_pos"], 32]
         elif source.location == "storage":
             length = ["add", ["sload", "_pos"], 32]
             pos_node = LLLnode.from_list(pos_node, typ=source.typ, location=source.location,)
         else:
    -        raise CompilerPanic(f"Unsupported location: {source.location}")
    +        raise CompilerPanic(f"Unsupported location: {source.location} to {destination.location}")
         if destination.location == "storage":
             destination = LLLnode.from_list(
                 destination, typ=destination.typ, location=destination.location,
    @@ -149,7 +163,7 @@ def make_byte_slice_copier(destination, source, length, max_length, pos=None):
                 [
                     "with",
                     "_l",
    -                max_length,
    +                max_length,  # CMC 20210917 shouldn't this just be length
                     ["pop", ["call", ["gas"], 4, 0, source, "_l", destination, "_l"]],
                 ],
                 typ=None,
    @@ -161,20 +175,19 @@ def make_byte_slice_copier(destination, source, length, max_length, pos=None):
         if source.value is None:
     
             if destination.location == "memory":
    +            # CMC 20210917 shouldn't this just be length
                 return mzero(destination, max_length)
     
             else:
                 loader = 0
         # Copy over data
    -    elif source.location == "memory":
    -        loader = ["mload", ["add", "_pos", ["mul", 32, ["mload", MemoryPositions.FREE_LOOP_INDEX]]]]
    -    elif source.location == "storage":
    -        loader = ["sload", ["add", "_pos", ["mload", MemoryPositions.FREE_LOOP_INDEX]]]
    -    elif source.location == "calldata":
    +    elif source.location in ("memory", "calldata", "code"):
             loader = [
    -            "calldataload",
    +            load_op(source.location),
                 ["add", "_pos", ["mul", 32, ["mload", MemoryPositions.FREE_LOOP_INDEX]]],
             ]
    +    elif source.location == "storage":
    +        loader = ["sload", ["add", "_pos", ["mload", MemoryPositions.FREE_LOOP_INDEX]]]
         else:
             raise CompilerPanic(f"Unsupported location: {source.location}")
         # Where to paste it?
    @@ -227,16 +240,19 @@ def make_byte_slice_copier(destination, source, length, max_length, pos=None):
     def byte_array_to_num(
         arg, expr, out_type, offset=32,
     ):
    -    if arg.location == "memory":
    -        lengetter = LLLnode.from_list(["mload", "_sub"], typ=BaseType("int256"))
    -        first_el_getter = LLLnode.from_list(["mload", ["add", 32, "_sub"]], typ=BaseType("int256"))
    -    elif arg.location == "storage":
    +    if arg.location == "storage":
             lengetter = LLLnode.from_list(["sload", "_sub"], typ=BaseType("int256"))
             first_el_getter = LLLnode.from_list(["sload", ["add", 1, "_sub"]], typ=BaseType("int256"))
    +    else:
    +        op = load_op(arg.location)
    +        lengetter = LLLnode.from_list([op, "_sub"], typ=BaseType("int256"))
    +        first_el_getter = LLLnode.from_list([op, ["add", 32, "_sub"]], typ=BaseType("int256"))
    +
         if out_type == "int128":
             result = int128_clamp(["div", "_el1", ["exp", 256, ["sub", 32, "_len"]]])
         elif out_type in ("int256", "uint256"):
             result = ["div", "_el1", ["exp", 256, ["sub", offset, "_len"]]]
    +    # TODO decimal clamp?
         return LLLnode.from_list(
             [
                 "with",
    @@ -256,13 +272,7 @@ def byte_array_to_num(
     
     def get_bytearray_length(arg):
         typ = BaseType("uint256")
    -    if arg.location == "memory":
    -        return LLLnode.from_list(["mload", arg], typ=typ)
    -    elif arg.location == "storage":
    -        return LLLnode.from_list(["sload", arg], typ=typ)
    -    elif arg.location == "calldata":
    -        return LLLnode.from_list(["calldataload", arg], typ=typ)
    -    raise CompilerPanic("unreachable", arg)  # pragma: no test
    +    return LLLnode.from_list([load_op(arg.location), arg], typ=typ)
     
     
     def getpos(node):
    @@ -274,11 +284,57 @@ def getpos(node):
         )
     
     
    +def _add_ofst(loc, ofst):
    +    if isinstance(loc.value, int) and isinstance(ofst, int):
    +        ret = loc.value + ofst
    +    else:
    +        ret = ["add", loc, ofst]
    +    return LLLnode.from_list(ret, location=loc.location, encoding=loc.encoding)
    +
    +
     # Take a value representing a memory or storage location, and descend down to
     # an element or member variable
    +# This is analogous (but not necessarily equivalent to) getelementptr in LLVM.
    +# TODO refactor / streamline this code, especially the ABI decoding
     @type_check_wrapper
    -def add_variable_offset(parent, key, pos, array_bounds_check=True):
    +def get_element_ptr(parent, key, pos, array_bounds_check=True):
    +    # TODO rethink this circular import
    +    from vyper.old_codegen.abi import abi_type_of
    +
         typ, location = parent.typ, parent.location
    +
    +    def _abi_helper(member_t, ofst, clamp=True):
    +        member_abi_t = abi_type_of(member_t)
    +        ofst_lll = _add_ofst(parent, ofst)
    +
    +        if member_abi_t.is_dynamic():
    +            # double dereference, according to ABI spec
    +            # TODO optimize special case: first dynamic item
    +            # offset is statically known.
    +            ofst_lll = _add_ofst(parent, unwrap_location(ofst_lll))
    +
    +        x = LLLnode.from_list(
    +            ["ofst"], typ=member_t, location=parent.location, annotation=f"&({typ}->{member_t})"
    +        )
    +
    +        if clamp and _needs_clamp(member_t, parent.encoding):
    +            # special handling for unsanitized external data that need clamping
    +            # TODO optimize me. this results in a double dereference because
    +            # it returns a pointer and not a value. probably the best thing
    +            # is to move the clamp to make_setter
    +            ret = ["with", x, ofst_lll, ["seq", clamp_basetype(x), x]]
    +        else:
    +            ret = ofst_lll
    +
    +        return LLLnode.from_list(
    +            ret,
    +            typ=member_t,
    +            location=parent.location,
    +            encoding=parent.encoding,
    +            pos=pos,
    +            # annotation=f"({parent.typ})[{key.typ}]",
    +        )
    +
         if isinstance(typ, TupleLike):
             if isinstance(typ, StructType):
                 subtype = typ.members[key]
    @@ -294,62 +350,47 @@ def add_variable_offset(parent, key, pos, array_bounds_check=True):
             if parent.value is None:
                 return LLLnode.from_list(None, typ=subtype)
     
    +        if parent.value == "multi":
    +            assert parent.encoding != Encoding.ABI, "no abi-encoded literals"
    +            return parent.args[index]
    +
    +        if parent.encoding in (Encoding.ABI, Encoding.JSON_ABI):
    +            if parent.location == "storage":
    +                raise CompilerPanic("storage variables should not be abi encoded")
    +
    +            # parent_abi_t = abi_type_of(parent.typ)
    +            member_t = typ.members[attrs[index]]
    +
    +            ofst = 0  # offset from parent start
    +
    +            for i in range(index):
    +                member_abi_t = abi_type_of(typ.members[attrs[i]])
    +                ofst += member_abi_t.embedded_static_size()
    +
    +            return _abi_helper(member_t, ofst)
    +
             if location == "storage":
                 # for arrays and structs, calculate the storage slot by adding an offset
                 # of [index value being accessed] * [size of each item within the sequence]
                 offset = 0
                 for i in range(index):
                     offset += get_size_of_type(typ.members[attrs[i]])
    -            return LLLnode.from_list(["add", parent, offset], typ=subtype, location="storage",)
    -        elif location == "storage_prehashed":
                 return LLLnode.from_list(
    -                ["add", parent, LLLnode.from_list(index, annotation=annotation)],
    -                typ=subtype,
    -                location="storage",
    +                ["add", parent, offset], typ=subtype, location="storage", pos=pos,
                 )
    -        elif location in ("calldata", "memory"):
    +
    +        elif location in ("calldata", "memory", "code"):
                 offset = 0
                 for i in range(index):
                     offset += 32 * get_size_of_type(typ.members[attrs[i]])
                 return LLLnode.from_list(
    -                ["add", offset, parent],
    +                _add_ofst(parent, offset),
                     typ=typ.members[key],
                     location=location,
                     annotation=annotation,
    +                pos=pos,
                 )
     
    -    elif isinstance(typ, MappingType):
    -
    -        sub = None
    -        if isinstance(key.typ, ByteArrayLike):
    -            if isinstance(typ.keytype, ByteArrayLike) and (typ.keytype.maxlen >= key.typ.maxlen):
    -
    -                subtype = typ.valuetype
    -                if len(key.args[0].args) >= 3:  # handle bytes literal.
    -                    sub = LLLnode.from_list(
    -                        [
    -                            "seq",
    -                            key,
    -                            [
    -                                "sha3",
    -                                ["add", key.args[0].args[-1], 32],
    -                                ["mload", key.args[0].args[-1]],
    -                            ],
    -                        ]
    -                    )
    -                else:
    -                    value = key.args[0].value
    -                    if value == "add":
    -                        # special case, key is a bytes array within a tuple/struct
    -                        value = key.args[0]
    -                    sub = LLLnode.from_list(["sha3", ["add", value, 32], key])
    -        else:
    -            subtype = typ.valuetype
    -            sub = unwrap_location(key)
    -
    -        if sub is not None and location == "storage":
    -            return LLLnode.from_list(["sha3_64", parent, sub], typ=subtype, location="storage")
    -
         elif isinstance(typ, ListType) and is_base_type(key.typ, ("int128", "int256", "uint256")):
             subtype = typ.subtype
     
    @@ -371,156 +412,150 @@ def add_variable_offset(parent, key, pos, array_bounds_check=True):
                 # an array index, and the clamp will throw an error.
                 sub = ["uclamplt", k, typ.count]
     
    +        if parent.encoding in (Encoding.ABI, Encoding.JSON_ABI):
    +            if parent.location == "storage":
    +                raise CompilerPanic("storage variables should not be abi encoded")
    +
    +            member_t = typ.subtype
    +            member_abi_t = abi_type_of(member_t)
    +
    +            if key.typ.is_literal:
    +                # TODO this constant folding in LLL optimizer
    +                ofst = k.value * member_abi_t.embedded_static_size()
    +            else:
    +                ofst = ["mul", k, member_abi_t.embedded_static_size()]
    +
    +            return _abi_helper(member_t, ofst)
    +
             if location == "storage":
                 # storage slot determined as [initial storage slot] + [index] * [size of base type]
                 offset = get_size_of_type(subtype)
                 return LLLnode.from_list(
                     ["add", parent, ["mul", sub, offset]], typ=subtype, location="storage", pos=pos
                 )
    -        elif location == "storage_prehashed":
    -            return LLLnode.from_list(["add", parent, sub], typ=subtype, location="storage", pos=pos)
    -        elif location in ("calldata", "memory"):
    +        elif location in ("calldata", "memory", "code"):
                 offset = 32 * get_size_of_type(subtype)
                 return LLLnode.from_list(
                     ["add", ["mul", offset, sub], parent], typ=subtype, location=location, pos=pos
                 )
     
    +    elif isinstance(typ, MappingType):
    +        sub = None
    +        if isinstance(key.typ, ByteArrayLike):
    +            # CMC 20210916 pretty sure this is dead code. TODO double check
    +            if isinstance(typ.keytype, ByteArrayLike) and (typ.keytype.maxlen >= key.typ.maxlen):
    +                subtype = typ.valuetype
    +                if len(key.args[0].args) >= 3:  # handle bytes literal.
    +                    sub = LLLnode.from_list(
    +                        [
    +                            "seq",
    +                            key,
    +                            [
    +                                "sha3",
    +                                ["add", key.args[0].args[-1], 32],
    +                                ["mload", key.args[0].args[-1]],
    +                            ],
    +                        ]
    +                    )
    +                else:
    +                    value = key.args[0].value
    +                    if value == "add":
    +                        # special case, key is a bytes array within a tuple/struct
    +                        value = key.args[0]
    +                    sub = LLLnode.from_list(["sha3", ["add", value, 32], key])
    +        else:
    +            subtype = typ.valuetype
    +            sub = unwrap_location(key)
    +
    +        if sub is not None and location == "storage":
    +            return LLLnode.from_list(["sha3_64", parent, sub], typ=subtype, location="storage")
    +
    +
    +def load_op(location):
    +    if location == "memory":
    +        return "mload"
    +    if location == "storage":
    +        return "sload"
    +    if location == "calldata":
    +        return "calldataload"
    +    if location == "code":
    +        return "codeload"
    +    raise CompilerPanic(f"unreachable {location}")  # pragma: no test
    +
     
     # Unwrap location
     def unwrap_location(orig):
    -    if orig.location == "memory":
    -        return LLLnode.from_list(["mload", orig], typ=orig.typ)
    -    elif orig.location == "storage":
    -        return LLLnode.from_list(["sload", orig], typ=orig.typ)
    -    elif orig.location == "calldata":
    -        return LLLnode.from_list(["calldataload", orig], typ=orig.typ)
    +    if orig.location in ("memory", "storage", "calldata", "code"):
    +        return LLLnode.from_list([load_op(orig.location), orig], typ=orig.typ)
         else:
    +        # CMC 20210909 TODO double check if this branch can be removed
             # handle None value inserted by `empty`
             if orig.value is None:
                 return LLLnode.from_list(0, typ=orig.typ)
             return orig
     
     
    -# Pack function arguments for a call
    -@type_check_wrapper
    -def pack_arguments(signature, args, context, stmt_expr, is_external_call):
    -    pos = getpos(stmt_expr)
    -    setters = []
    -    staticarray_offset = 0
    -
    -    maxlen = sum([get_size_of_type(arg.typ) for arg in signature.args]) * 32
    -    if is_external_call:
    -        maxlen += 32
    -
    -    placeholder_typ = ByteArrayType(maxlen=maxlen)
    -    placeholder = context.new_internal_variable(placeholder_typ)
    -    if is_external_call:
    -        setters.append(["mstore", placeholder, signature.method_id])
    -        placeholder += 32
    -
    -    if len(signature.args) != len(args):
    -        return
    -
    -    # check for dynamic-length types
    -    dynamic_remaining = len([i for i in signature.args if isinstance(i.typ, ByteArrayLike)])
    -    needpos = bool(dynamic_remaining)
    -
    -    for i, (arg, typ) in enumerate(zip(args, [arg.typ for arg in signature.args])):
    -        if isinstance(typ, BaseType):
    -            setters.append(
    -                make_setter(
    -                    LLLnode.from_list(placeholder + staticarray_offset + i * 32, typ=typ,),
    -                    arg,
    -                    "memory",
    -                    pos=pos,
    -                    in_function_call=True,
    -                )
    -            )
    -
    -        elif isinstance(typ, ByteArrayLike):
    -            dynamic_remaining -= 1
    -            setters.append(["mstore", placeholder + staticarray_offset + i * 32, "_poz"])
    -            arg_copy = LLLnode.from_list("_s", typ=arg.typ, location=arg.location)
    -            target = LLLnode.from_list(["add", placeholder, "_poz"], typ=typ, location="memory",)
    -            pos_setter = "pass"
    -
    -            # if `arg.value` is None, this is a call to `empty()`
    -            # if `arg.typ.maxlen` is 0, this is a literal "" or b""
    -            if arg.value is None or arg.typ.maxlen == 0:
    -                if dynamic_remaining:
    -                    # only adjust the dynamic pointer if this is not the last dynamic type
    -                    pos_setter = ["set", "_poz", ["add", "_poz", 64]]
    -                setters.append(["seq", mzero(target, 64), pos_setter])
    -            else:
    -                if dynamic_remaining:
    -                    pos_setter = [
    -                        "set",
    -                        "_poz",
    -                        ["add", 32, ["ceil32", ["add", "_poz", get_bytearray_length(arg_copy)]]],
    -                    ]
    -                setters.append(
    -                    [
    -                        "with",
    -                        "_s",
    -                        arg,
    -                        ["seq", make_byte_array_copier(target, arg_copy, pos), pos_setter],
    -                    ]
    -                )
    -
    -        elif isinstance(typ, (StructType, ListType)):
    -            if has_dynamic_data(typ):
    -                return
    -            target = LLLnode.from_list(
    -                [placeholder + staticarray_offset + i * 32], typ=typ, location="memory",
    -            )
    -            setters.append(make_setter(target, arg, "memory", pos=pos))
    -            staticarray_offset += 32 * (get_size_of_type(typ) - 1)
    -
    -        else:
    -            return
    -
    -    if is_external_call:
    -        returner = [[placeholder - 4]]
    -        inargsize = placeholder_typ.maxlen - 28
    -    else:
    -        # internal call does not use a returner or adjust max length for signature
    -        returner = []
    -        inargsize = placeholder_typ.maxlen
    -
    -    if needpos:
    -        return (
    -            LLLnode.from_list(
    -                ["with", "_poz", len(args) * 32 + staticarray_offset, ["seq"] + setters + returner],
    -                typ=placeholder_typ,
    -                location="memory",
    -            ),
    -            inargsize,
    -            placeholder,
    -        )
    -    else:
    -        return (
    -            LLLnode.from_list(["seq"] + setters + returner, typ=placeholder_typ, location="memory"),
    -            inargsize,
    -            placeholder,
    -        )
    -
    -
     def _make_array_index_setter(target, target_token, pos, location, offset):
         if location == "memory" and isinstance(target.value, int):
             offset = target.value + 32 * get_size_of_type(target.typ.subtype) * offset
             return LLLnode.from_list([offset], typ=target.typ.subtype, location=location, pos=pos)
         else:
    -        return add_variable_offset(
    +        return get_element_ptr(
                 target_token,
                 LLLnode.from_list(offset, typ="int256"),
                 pos=pos,
                 array_bounds_check=False,
             )
     
     
    +# utility function, constructs an LLL tuple out of a list of LLL nodes
    +def lll_tuple_from_args(args):
    +    typ = TupleType([x.typ for x in args])
    +    return LLLnode.from_list(["multi"] + [x for x in args], typ=typ)
    +
    +
    +def _needs_external_call_wrap(lll_typ):
    +    # for calls to ABI conforming contracts.
    +    # according to the ABI spec, return types are ALWAYS tuples even
    +    # if only one element is being returned.
    +    # https://solidity.readthedocs.io/en/latest/abi-spec.html#function-selector-and-argument-encoding
    +    # "and the return values v_1, ..., v_k of f are encoded as
    +    #
    +    #    enc((v_1, ..., v_k))
    +    #    i.e. the values are combined into a tuple and encoded.
    +    # "
    +    # therefore, wrap it in a tuple if it's not already a tuple.
    +    # for example, `bytes` is returned as abi-encoded (bytes,)
    +    # and `(bytes,)` is returned as abi-encoded ((bytes,),)
    +    # In general `-> X` gets returned as (X,)
    +    # including structs. MyStruct is returned as abi-encoded (MyStruct,).
    +    # (Sorry this is so confusing. I didn't make these rules.)
    +
    +    return not (isinstance(lll_typ, TupleType) and len(lll_typ.members) > 1)
    +
    +
    +def calculate_type_for_external_return(lll_typ):
    +    if _needs_external_call_wrap(lll_typ):
    +        return TupleType([lll_typ])
    +    return lll_typ
    +
    +
    +def wrap_value_for_external_return(lll_val):
    +    # used for LHS promotion
    +    if _needs_external_call_wrap(lll_val.typ):
    +        return lll_tuple_from_args([lll_val])
    +    else:
    +        return lll_val
    +
    +
    +def set_type_for_external_return(lll_val):
    +    # used for RHS promotion
    +    lll_val.typ = calculate_type_for_external_return(lll_val.typ)
    +
    +
     # Create an x=y statement, where the types may be compound
     @type_check_wrapper
    -def make_setter(left, right, location, pos, in_function_call=False):
    +def make_setter(left, right, location, pos):
         # Basic types
         if isinstance(left.typ, BaseType):
             right = unwrap_location(right)
    @@ -546,14 +581,8 @@ def make_setter(left, right, location, pos, in_function_call=False):
     
             left_token = LLLnode.from_list("_L", typ=left.typ, location=left.location)
             # If the right side is a literal
    -        if right.value in ["multi", "seq_unchecked"] and right.typ.is_literal:
    -            if right.value == "seq_unchecked":
    -                # when the LLL is `seq_unchecked`, this is a literal where one or
    -                # more values must be pre-processed to avoid memory corruption
    -                subs = right.args[:-1]
    -                right = right.args[-1]
    -            else:
    -                subs = []
    +        if right.value == "multi":
    +            subs = []
                 for i in range(left.typ.count):
                     lhs_setter = _make_array_index_setter(left, left_token, pos, location, i)
                     subs.append(make_setter(lhs_setter, right.args[i], location, pos=pos,))
    @@ -571,7 +600,7 @@ def make_setter(left, right, location, pos, in_function_call=False):
                 for i in range(left.typ.count):
                     subs.append(
                         make_setter(
    -                        add_variable_offset(
    +                        get_element_ptr(
                                 left_token,
                                 LLLnode.from_list(i, typ="int256"),
                                 pos=pos,
    @@ -585,7 +614,9 @@ def make_setter(left, right, location, pos, in_function_call=False):
                 return LLLnode.from_list(["with", "_L", left, ["seq"] + subs], typ=None)
             # If the right side is a variable
             else:
    -            right_token = LLLnode.from_list("_R", typ=right.typ, location=right.location)
    +            right_token = LLLnode.from_list(
    +                "_R", typ=right.typ, location=right.location, encoding=right.encoding
    +            )
                 subs = []
                 for i in range(left.typ.count):
                     lhs_setter = _make_array_index_setter(left, left_token, pos, left.location, i)
    @@ -620,7 +651,10 @@ def make_setter(left, right, location, pos, in_function_call=False):
             left_token = LLLnode.from_list("_L", typ=left.typ, location=left.location)
             keyz = left.typ.tuple_keys()
     
    -        # If the left side is a literal
    +        if len(keyz) == 0:
    +            return LLLnode.from_list(["pass"])
    +
    +        # If the left side is complex
             if left.value == "multi":
                 locations = [arg.location for arg in left.args]
             else:
    @@ -630,6 +664,8 @@ def make_setter(left, right, location, pos, in_function_call=False):
             if right.value == "multi":
                 if len(right.args) != len(keyz):
                     return
    +            if left.value == "multi":
    +                left_token = left
                 # get the RHS arguments into a dict because
                 # they are not guaranteed to be in the same order
                 # the LHS keys.
    @@ -638,10 +674,7 @@ def make_setter(left, right, location, pos, in_function_call=False):
                 for (key, loc) in zip(keyz, locations):
                     subs.append(
                         make_setter(
    -                        add_variable_offset(left_token, key, pos=pos),
    -                        right_args[key],
    -                        loc,
    -                        pos=pos,
    +                        get_element_ptr(left_token, key, pos=pos), right_args[key], loc, pos=pos,
                         )
                     )
                 return LLLnode.from_list(["with", "_L", left, ["seq"] + subs], typ=None)
    @@ -657,40 +690,44 @@ def make_setter(left, right, location, pos, in_function_call=False):
                 for key, loc in zip(keyz, locations):
                     subs.append(
                         make_setter(
    -                        add_variable_offset(left_token, key, pos=pos),
    +                        get_element_ptr(left_token, key, pos=pos),
                             LLLnode.from_list(None, typ=right.typ.members[key]),
                             loc,
                             pos=pos,
                         )
                     )
                 return LLLnode.from_list(["with", "_L", left, ["seq"] + subs], typ=None)
    -        # If tuple assign.
    -        elif isinstance(left.typ, TupleType) and isinstance(right.typ, TupleType):
    +        # literal tuple assign.
    +        elif (
    +            isinstance(left.typ, TupleType)
    +            and left.value == "multi"
    +            and isinstance(right.typ, TupleType)
    +        ):
                 subs = []
                 for var_arg in left.args:
    -                if var_arg.location == "calldata":
    +                if var_arg.location in ("calldata", "code"):
                         return
     
    -            right_token = LLLnode.from_list("_R", typ=right.typ, location=right.location)
    +            right_token = LLLnode.from_list(
    +                "_R", typ=right.typ, location=right.location, encoding=right.encoding
    +            )
                 for left_arg, key, loc in zip(left.args, keyz, locations):
                     subs.append(
    -                    make_setter(
    -                        left_arg, add_variable_offset(right_token, key, pos=pos), loc, pos=pos
    -                    )
    +                    make_setter(left_arg, get_element_ptr(right_token, key, pos=pos), loc, pos=pos)
                     )
     
    -            return LLLnode.from_list(
    -                ["with", "_R", right, ["seq"] + subs], typ=None, annotation="Tuple assignment",
    -            )
    -        # If the left side is a variable i.e struct type
    +            return LLLnode.from_list(["with", "_R", right, ["seq"] + subs], typ=None)
    +        # general case
             else:
                 subs = []
    -            right_token = LLLnode.from_list("_R", typ=right.typ, location=right.location)
    +            right_token = LLLnode.from_list(
    +                "_R", typ=right.typ, location=right.location, encoding=right.encoding
    +            )
                 for typ, loc in zip(keyz, locations):
                     subs.append(
                         make_setter(
    -                        add_variable_offset(left_token, typ, pos=pos),
    -                        add_variable_offset(right_token, typ, pos=pos),
    +                        get_element_ptr(left_token, typ, pos=pos),
    +                        get_element_ptr(right_token, typ, pos=pos),
                             loc,
                             pos=pos,
                         )
    @@ -700,6 +737,7 @@ def make_setter(left, right, location, pos, in_function_call=False):
                 )
     
     
    +# TODO move return checks to vyper/semantics/validation
     def is_return_from_function(node):
         if isinstance(node, vy_ast.Expr) and node.get("value.func.id") == "selfdestruct":
             return True
    @@ -735,21 +773,6 @@ def _check_return_body(node, node_list):
                 )
     
     
    -def _return_check(node):
    -    if is_return_from_function(node):
    -        return True
    -    elif isinstance(node, list):
    -        return any(_return_check(stmt) for stmt in node)
    -    elif isinstance(node, vy_ast.If):
    -        if_body_check = _return_check(node.body)
    -        else_body_check = _return_check(node.orelse)
    -        if if_body_check and else_body_check:  # both side need to match.
    -            return True
    -        else:
    -            return False
    -    return False
    -
    -
     def mzero(dst, nbytes):
         # calldatacopy from past-the-end gives zero bytes.
         # cf. YP H.2 (ops section) with CALLDATACOPY spec.
    @@ -775,3 +798,80 @@ def zero_pad(bytez_placeholder):
             ["with", "len", len_, ["with", "dst", dst, mzero("dst", num_zero_bytes)]],
             annotation="Zero pad",
         )
    +
    +
    +# convenience rewrites for shr/sar/shl
    +def _shr(x, bits):
    +    if version_check(begin="constantinople"):
    +        return ["shr", bits, x]
    +    return ["div", x, ["exp", 2, bits]]
    +
    +
    +def _sar(x, bits):
    +    if version_check(begin="constantinople"):
    +        return ["sar", bits, x]
    +
    +    # emulate for older arches. keep in mind note from EIP 145:
    +    # This is not equivalent to PUSH1 2 EXP SDIV, since it rounds
    +    # differently. See SDIV(-1, 2) == 0, while SAR(-1, 1) == -1.
    +    return ["sdiv", ["add", ["slt", x, 0], x], ["exp", 2, bits]]
    +
    +
    +def _needs_clamp(t, encoding):
    +    assert encoding in (Encoding.ABI, Encoding.JSON_ABI)
    +    if isinstance(t, ByteArrayLike):
    +        if encoding == Encoding.JSON_ABI:
    +            # don't have bytestring size bound from json, don't clamp
    +            return False
    +        return True
    +    if isinstance(t, BaseType) and t.typ not in ("int256", "uint256", "bytes32"):
    +        return True
    +    return False
    +
    +
    +# clampers for basetype
    +@type_check_wrapper
    +def clamp_basetype(lll_node):
    +    t = lll_node.typ
    +    if isinstance(t, ByteArrayLike):
    +        return ["assert", ["le", get_bytearray_length(lll_node), t.maxlen]]
    +    if isinstance(t, BaseType):
    +        lll_node = unwrap_location(lll_node)
    +        if t.typ in ("int128"):
    +            return int_clamp(lll_node, 128, signed=True)
    +        if t.typ in ("decimal"):
    +            return [
    +                "clamp",
    +                ["mload", MemoryPositions.MINDECIMAL],
    +                lll_node,
    +                ["mload", MemoryPositions.MAXDECIMAL],
    +            ]
    +
    +        if t.typ in ("address",):
    +            return int_clamp(lll_node, 160)
    +        if t.typ in ("bool",):
    +            return int_clamp(lll_node, 1)
    +        if t.typ in ("int256", "uint256", "bytes32"):
    +            return ["pass"]  # special case, no clamp
    +    return  # raises
    +
    +
    +def int_clamp(lll_node, bits, signed=False):
    +    """Generalized clamper for integer types. Takes the number of bits,
    +       whether it's signed, and returns an LLL node which checks it is
    +       in bounds.
    +    """
    +    if bits >= 256:
    +        raise CompilerPanic("shouldn't clamp", lll_node)
    +    if signed:
    +        # example for bits==128:
    +        # if _val is in bounds,
    +        # _val >>> 127 == 0 for positive _val
    +        # _val >>> 127 == -1 for negative _val
    +        # -1 and 0 are the only numbers which are unchanged by sar,
    +        # so sar'ing (_val>>>127) one more bit should leave it unchanged.
    +        ret = ["with", "x", lll_node, ["assert", ["eq", _sar("x", bits - 1), _sar("x", bits)]]]
    +    else:
    +        ret = ["assert", ["iszero", _shr(lll_node, bits)]]
    +
    +    return LLLnode.from_list(ret, annotation=f"int_clamp {lll_node.typ}")
    
  • vyper/old_codegen/return_.py+64 150 modified
    @@ -1,158 +1,72 @@
    -from vyper import ast as vy_ast
    -from vyper.old_codegen.function_definitions.utils import get_nonreentrant_lock
    +from typing import Any, Optional
    +
    +from vyper.old_codegen.abi import abi_encode, abi_type_of
    +from vyper.old_codegen.context import Context
     from vyper.old_codegen.lll_node import LLLnode
    -from vyper.old_codegen.parser_utils import getpos, make_setter
    -from vyper.old_codegen.types import BaseType, TupleType, get_size_of_type
    +from vyper.old_codegen.parser_utils import (
    +    getpos,
    +    make_setter,
    +    wrap_value_for_external_return,
    +)
    +from vyper.old_codegen.types import get_type_for_exact_size
     from vyper.old_codegen.types.check import check_assign
    -from vyper.utils import MemoryPositions
     
    -from .abi import abi_encode, abi_type_of, ensure_tuple
    +Stmt = Any  # mypy kludge
     
     
    -# Generate return code for stmt
    -def make_return_stmt(stmt, context, begin_pos, _size, loop_memory_position=None):
    -    # TODO check this out
    -    func_type = stmt.get_ancestor(vy_ast.FunctionDef)._metadata["type"]
    -    _, nonreentrant_post = get_nonreentrant_lock(func_type)
    +# Generate code for return stmt
    +def make_return_stmt(lll_val: LLLnode, stmt: Any, context: Context) -> Optional[LLLnode]:
    +
    +    sig = context.sig
    +
    +    jump_to_exit = ["goto", sig.exit_sequence_label]
    +
    +    _pos = getpos(stmt)
    +
    +    if context.return_type is None:
    +        if stmt.value is not None:
    +            return None  # triggers an exception
     
    -    if context.is_internal:
    -        if loop_memory_position is None:
    -            loop_memory_position = context.new_internal_variable(BaseType("uint256"))
    -
    -        # Make label for stack push loop.
    -        label_id = "_".join([str(x) for x in (context.method_id, stmt.lineno, stmt.col_offset)])
    -        exit_label = f"make_return_loop_exit_{label_id}"
    -        start_label = f"make_return_loop_start_{label_id}"
    -
    -        # Push prepared data onto the stack,
    -        # in reverse order so it can be popped of in order.
    -        if isinstance(begin_pos, int) and isinstance(_size, int):
    -            # static values, unroll the mloads instead.
    -            mloads = [["mload", pos] for pos in range(begin_pos, _size, 32)]
    -        else:
    -            mloads = [
    -                "seq_unchecked",
    -                ["mstore", loop_memory_position, _size],
    -                ["label", start_label],
    -                [  # maybe exit loop / break.
    -                    "if",
    -                    ["le", ["mload", loop_memory_position], 0],
    -                    ["goto", exit_label],
    -                ],
    -                [  # push onto stack
    -                    "mload",
    -                    ["add", begin_pos, ["sub", ["mload", loop_memory_position], 32]],
    -                ],
    -                [  # decrement i by 32.
    -                    "mstore",
    -                    loop_memory_position,
    -                    ["sub", ["mload", loop_memory_position], 32],
    -                ],
    -                ["goto", start_label],
    -                ["label", exit_label],
    -            ]
    -
    -        # if we are in a for loop, we have to exit prior to returning
    -        exit_repeater = ["exit_repeater"] if context.forvars else []
    -
    -        return (
    -            ["seq_unchecked"]
    -            + exit_repeater
    -            + mloads
    -            + nonreentrant_post
    -            + [["jump", ["mload", context.callback_ptr]]]
    -        )
         else:
    -        return ["seq_unchecked"] + nonreentrant_post + [["return", begin_pos, _size]]
    -
    -
    -# Generate code for returning a tuple or struct.
    -def gen_tuple_return(stmt, context, sub):
    -    abi_typ = abi_type_of(context.return_type)
    -    # according to the ABI, return types are ALWAYS tuples even if
    -    # only one element is being returned.
    -    # https://solidity.readthedocs.io/en/latest/abi-spec.html#function-selector-and-argument-encoding
    -    # "and the return values v_1, ..., v_k of f are encoded as
    -    #
    -    #    enc((v_1, ..., v_k))
    -    #    i.e. the values are combined into a tuple and encoded.
    -    # "
    -    # therefore, wrap it in a tuple if it's not already a tuple.
    -    # (big difference between returning `(bytes,)` and `bytes`.
    -    abi_typ = ensure_tuple(abi_typ)
    -    abi_bytes_needed = abi_typ.static_size() + abi_typ.dynamic_size_bound()
    -    dst = context.memory_allocator.expand_memory(abi_bytes_needed)
    -    return_buffer = LLLnode(
    -        dst, location="memory", annotation="return_buffer", typ=context.return_type
    -    )
    -
    -    check_assign(return_buffer, sub, pos=getpos(stmt))
    -
    -    if sub.value == "multi":
    -
    -        if isinstance(context.return_type, TupleType) and not abi_typ.dynamic_size_bound():
    -            # for tuples where every value is of the same type and a fixed length,
    -            # we can simplify the encoding by using make_setter, since
    -            # our memory encoding happens to be identical to the ABI
    -            # encoding.
    -            new_sub = LLLnode.from_list(
    -                context.new_internal_variable(context.return_type),
    -                typ=context.return_type,
    -                location="memory",
    -            )
    -            setter = make_setter(new_sub, sub, "memory", pos=getpos(stmt))
    -            return LLLnode.from_list(
    -                [
    -                    "seq",
    -                    setter,
    -                    make_return_stmt(
    -                        stmt, context, new_sub, get_size_of_type(context.return_type) * 32,
    -                    ),
    -                ],
    -                typ=None,
    -                pos=getpos(stmt),
    -            )
    -
    -        # in case of multi we can't create a variable to store location of the return expression
    -        # as multi can have data from multiple location like store, calldata etc
    -        encode_out = abi_encode(return_buffer, sub, pos=getpos(stmt), returns_len=True)
    -        load_return_len = ["mload", MemoryPositions.FREE_VAR_SPACE]
    -        os = [
    -            "seq",
    -            ["mstore", MemoryPositions.FREE_VAR_SPACE, encode_out],
    -            make_return_stmt(stmt, context, return_buffer, load_return_len),
    -        ]
    -        return LLLnode.from_list(os, typ=None, pos=getpos(stmt), valency=0)
    -
    -    # for tuple return types where a function is called inside the tuple, we
    -    # process the calls prior to encoding the return data
    -    if sub.value == "seq_unchecked" and sub.args[-1].value == "multi":
    -        encode_out = abi_encode(return_buffer, sub.args[-1], pos=getpos(stmt), returns_len=True)
    -        load_return_len = ["mload", MemoryPositions.FREE_VAR_SPACE]
    -        os = (
    -            ["seq"]
    -            + sub.args[:-1]
    -            + [
    -                ["mstore", MemoryPositions.FREE_VAR_SPACE, encode_out],
    -                make_return_stmt(stmt, context, return_buffer, load_return_len),
    -            ]
    +        # sanity typecheck
    +        _tmp = LLLnode("fake node", location="memory", typ=context.return_type)
    +        check_assign(_tmp, lll_val, _pos)
    +
    +    # helper function
    +    def finalize(fill_return_buffer):
    +        # do NOT bypass this. jump_to_exit may do important function cleanup.
    +        fill_return_buffer = LLLnode.from_list(
    +            fill_return_buffer, annotation=f"fill return buffer {sig._lll_identifier}"
             )
    -        return LLLnode.from_list(os, typ=None, pos=getpos(stmt), valency=0)
    -
    -    # for all othe cases we are creating a stack variable named sub_loc to store the  location
    -    # of the return expression. This is done so that the return expression does not get evaluated
    -    # abi-encode uses a function named o_list which evaluate the expression multiple times
    -    sub_loc = LLLnode("sub_loc", typ=sub.typ, location=sub.location)
    -    encode_out = abi_encode(return_buffer, sub_loc, pos=getpos(stmt), returns_len=True)
    -    load_return_len = ["mload", MemoryPositions.FREE_VAR_SPACE]
    -    os = [
    -        "with",
    -        "sub_loc",
    -        sub,
    -        [
    -            "seq",
    -            ["mstore", MemoryPositions.FREE_VAR_SPACE, encode_out],
    -            make_return_stmt(stmt, context, return_buffer, load_return_len),
    -        ],
    -    ]
    -    return LLLnode.from_list(os, typ=None, pos=getpos(stmt), valency=0)
    +        cleanup_loops = "exit_repeater" if context.forvars else "pass"
    +        return LLLnode.from_list(
    +            ["seq_unchecked", cleanup_loops, fill_return_buffer, jump_to_exit], typ=None, pos=_pos,
    +        )
    +
    +    if context.return_type is None:
    +        return finalize(["pass"])
    +
    +    if context.is_internal:
    +        dst = LLLnode.from_list(["return_buffer"], typ=context.return_type, location="memory")
    +        fill_return_buffer = [
    +            "with",
    +            dst,
    +            "pass",  # return_buffer is passed on the stack by caller
    +            make_setter(dst, lll_val, location="memory", pos=_pos),
    +        ]
    +
    +        return finalize(fill_return_buffer)
    +
    +    else:  # return from external function
    +
    +        lll_val = wrap_value_for_external_return(lll_val)
    +
    +        maxlen = abi_type_of(context.return_type).size_bound()
    +        return_buffer_ofst = context.new_internal_variable(get_type_for_exact_size(maxlen))
    +
    +        # encode_out is cleverly a sequence which does the abi-encoding and
    +        # also returns the length of the output as a stack element
    +        encode_out = abi_encode(return_buffer_ofst, lll_val, pos=_pos, returns_len=True)
    +
    +        # fill the return buffer and push the location and length onto the stack
    +        return finalize(["seq_unchecked", encode_out, return_buffer_ofst])
    
  • vyper/old_codegen/self_call.py+78 256 modified
    @@ -1,72 +1,48 @@
    -import itertools
    +from vyper.exceptions import StateAccessViolation, StructureException
    +from vyper.old_codegen.lll_node import LLLnode, push_label_to_stack
    +from vyper.old_codegen.parser_utils import getpos, make_setter
    +from vyper.old_codegen.types import TupleType
     
    -from vyper.ast.signatures.function_signature import FunctionSignature
    -from vyper.exceptions import (
    -    StateAccessViolation,
    -    StructureException,
    -    TypeCheckFailure,
    -)
    -from vyper.old_codegen.abi import abi_decode
    -from vyper.old_codegen.lll_node import LLLnode
    -from vyper.old_codegen.parser_utils import getpos, pack_arguments
    -from vyper.old_codegen.types import (
    -    BaseType,
    -    ByteArrayLike,
    -    ListType,
    -    TupleLike,
    -    get_size_of_type,
    -    get_static_size_of_type,
    -    has_dynamic_data,
    -)
    +_label_counter = 0
     
     
    -def _call_make_placeholder(stmt_expr, context, sig):
    -    if sig.output_type is None:
    -        return 0, 0, 0
    +# TODO a more general way of doing this
    +def _generate_label(name: str) -> str:
    +    global _label_counter
    +    _label_counter += 1
    +    return f"label{_label_counter}"
     
    -    output_placeholder = context.new_internal_variable(typ=sig.output_type)
    -    output_size = get_size_of_type(sig.output_type) * 32
     
    -    if isinstance(sig.output_type, BaseType):
    -        returner = output_placeholder
    -    elif isinstance(sig.output_type, ByteArrayLike):
    -        returner = output_placeholder
    -    elif isinstance(sig.output_type, TupleLike):
    -        # incase of struct we need to decode the output and then return it
    -        returner = ["seq"]
    -        decoded_placeholder = context.new_internal_variable(typ=sig.output_type)
    -        decoded_node = LLLnode(decoded_placeholder, typ=sig.output_type, location="memory")
    -        output_node = LLLnode(output_placeholder, typ=sig.output_type, location="memory")
    -        returner.append(abi_decode(decoded_node, output_node))
    -        returner.extend([decoded_placeholder])
    -    elif isinstance(sig.output_type, ListType):
    -        returner = output_placeholder
    -    else:
    -        raise TypeCheckFailure(f"Invalid output type: {sig.output_type}")
    -    return output_placeholder, returner, output_size
    +def lll_for_self_call(stmt_expr, context):
    +    from vyper.old_codegen.expr import (
    +        Expr,  # TODO rethink this circular import
    +    )
     
    +    pos = getpos(stmt_expr)
     
    -def make_call(stmt_expr, context):
         # ** Internal Call **
         # Steps:
    -    # (x) push current local variables
    -    # (x) push arguments
    -    # (x) push jumpdest (callback ptr)
    -    # (x) jump to label
    -    # (x) pop return values
    -    # (x) pop local variables
    +    # - copy arguments into the soon-to-be callee
    +    # - allocate return buffer
    +    # - push jumpdest (callback ptr) and return buffer location
    +    # - jump to label
    +    # - (private function will fill return buffer and jump back)
     
    -    pop_local_vars = []
    -    push_local_vars = []
    -    pop_return_values = []
    -    push_args = []
         method_name = stmt_expr.func.attr
     
    -    # TODO check this out
    -    from vyper.old_codegen.expr import parse_sequence
    +    pos_args_lll = [Expr(x, context).lll_node for x in stmt_expr.args]
    +
    +    sig, kw_vals = context.lookup_internal_function(method_name, pos_args_lll)
    +
    +    kw_args_lll = [Expr(x, context).lll_node for x in kw_vals]
    +
    +    args_lll = pos_args_lll + kw_args_lll
     
    -    pre_init, expr_args = parse_sequence(stmt_expr, stmt_expr.args, context)
    -    sig = FunctionSignature.lookup_sig(context.sigs, method_name, expr_args, stmt_expr, context,)
    +    args_tuple_t = TupleType([x.typ for x in args_lll])
    +    args_as_tuple = LLLnode.from_list(["multi"] + [x for x in args_lll], typ=args_tuple_t)
    +
    +    # register callee to help calculate our starting frame offset
    +    context.register_callee(sig.frame_size)
     
         if context.is_constant() and sig.mutability not in ("view", "pure"):
             raise StateAccessViolation(
    @@ -75,217 +51,63 @@ def make_call(stmt_expr, context):
                 getpos(stmt_expr),
             )
     
    +    # TODO move me to type checker phase
         if not sig.internal:
             raise StructureException("Cannot call external functions via 'self'", stmt_expr)
     
    -    # Push local variables.
    -    var_slots = [(v.pos, v.size) for name, v in context.vars.items() if v.location == "memory"]
    -    if var_slots:
    -        var_slots.sort(key=lambda x: x[0])
    -
    -        if len(var_slots) > 10:
    -            # if memory is large enough, push and pop it via iteration
    -            mem_from, mem_to = var_slots[0][0], var_slots[-1][0] + var_slots[-1][1] * 32
    -            i_placeholder = context.new_internal_variable(BaseType("uint256"))
    -            local_save_ident = f"_{stmt_expr.lineno}_{stmt_expr.col_offset}"
    -            push_loop_label = "save_locals_start" + local_save_ident
    -            pop_loop_label = "restore_locals_start" + local_save_ident
    -            push_local_vars = [
    -                ["mstore", i_placeholder, mem_from],
    -                ["label", push_loop_label],
    -                ["mload", ["mload", i_placeholder]],
    -                ["mstore", i_placeholder, ["add", ["mload", i_placeholder], 32]],
    -                ["if", ["lt", ["mload", i_placeholder], mem_to], ["goto", push_loop_label]],
    -            ]
    -            pop_local_vars = [
    -                ["mstore", i_placeholder, mem_to - 32],
    -                ["label", pop_loop_label],
    -                ["mstore", ["mload", i_placeholder], "pass"],
    -                ["mstore", i_placeholder, ["sub", ["mload", i_placeholder], 32]],
    -                ["if", ["ge", ["mload", i_placeholder], mem_from], ["goto", pop_loop_label]],
    -            ]
    -        else:
    -            # for smaller memory, hardcode the mload/mstore locations
    -            push_mem_slots = []
    -            for pos, size in var_slots:
    -                push_mem_slots.extend([pos + i * 32 for i in range(size)])
    -
    -            push_local_vars = [["mload", pos] for pos in push_mem_slots]
    -            pop_local_vars = [["mstore", pos, "pass"] for pos in push_mem_slots[::-1]]
    +    return_label = _generate_label(f"{sig.internal_function_label}_call")
     
    -    # Push Arguments
    -    if expr_args:
    -        inargs, inargsize, arg_pos = pack_arguments(
    -            sig, expr_args, context, stmt_expr, is_external_call=False
    +    # allocate space for the return buffer
    +    # TODO allocate in stmt and/or expr.py
    +    return_buffer = (
    +        context.new_internal_variable(sig.return_type) if sig.return_type is not None else "pass"
    +    )
    +    return_buffer = LLLnode.from_list([return_buffer], annotation=f"{return_label}_return_buf")
    +
    +    # note: dst_tuple_t != args_tuple_t
    +    dst_tuple_t = TupleType([arg.typ for arg in sig.args])
    +    args_dst = LLLnode(sig.frame_start, typ=dst_tuple_t, location="memory")
    +
    +    # if one of the arguments is a self call, the argument
    +    # buffer could get borked. to prevent against that,
    +    # write args to a temporary buffer until all the arguments
    +    # are fully evaluated.
    +    if args_as_tuple.contains_self_call:
    +        copy_args = ["seq"]
    +        # TODO deallocate me
    +        tmp_args_buf = LLLnode(
    +            context.new_internal_variable(dst_tuple_t), typ=dst_tuple_t, location="memory",
             )
    -        push_args += [inargs]  # copy arguments first, to not mess up the push/pop sequencing.
    -
    -        static_arg_size = 32 * sum([get_static_size_of_type(arg.typ) for arg in expr_args])
    -        static_pos = int(arg_pos + static_arg_size)
    -        needs_dyn_section = any([has_dynamic_data(arg.typ) for arg in expr_args])
    -
    -        if needs_dyn_section:
    -            ident = f"push_args_{sig.method_id}_{stmt_expr.lineno}_{stmt_expr.col_offset}"
    -            start_label = ident + "_start"
    -            end_label = ident + "_end"
    -            i_placeholder = context.new_internal_variable(BaseType("uint256"))
    -
    -            # Calculate copy start position.
    -            # Given | static | dynamic | section in memory,
    -            # copy backwards so the values are in order on the stack.
    -            # We calculate i, the end of the whole encoded part
    -            # (i.e. the starting index for copy)
    -            # by taking ceil32(len<arg>) + offset<arg> + arg_pos
    -            # for the last dynamic argument and arg_pos is the start
    -            # the whole argument section.
    -            idx = 0
    -            for arg in expr_args:
    -                if isinstance(arg.typ, ByteArrayLike):
    -                    last_idx = idx
    -                idx += get_static_size_of_type(arg.typ)
    -            push_args += [
    -                [
    -                    "with",
    -                    "offset",
    -                    ["mload", arg_pos + last_idx * 32],
    -                    [
    -                        "with",
    -                        "len_pos",
    -                        ["add", arg_pos, "offset"],
    -                        [
    -                            "with",
    -                            "len_value",
    -                            ["mload", "len_pos"],
    -                            ["mstore", i_placeholder, ["add", "len_pos", ["ceil32", "len_value"]]],
    -                        ],
    -                    ],
    -                ]
    -            ]
    -            # loop from end of dynamic section to start of dynamic section,
    -            # pushing each element onto the stack.
    -            push_args += [
    -                ["label", start_label],
    -                ["if", ["lt", ["mload", i_placeholder], static_pos], ["goto", end_label]],
    -                ["mload", ["mload", i_placeholder]],
    -                ["mstore", i_placeholder, ["sub", ["mload", i_placeholder], 32]],  # decrease i
    -                ["goto", start_label],
    -                ["label", end_label],
    -            ]
    -
    -        # push static section
    -        push_args += [["mload", pos] for pos in reversed(range(arg_pos, static_pos, 32))]
    -    elif sig.args:
    -        raise StructureException(
    -            f"Wrong number of args for: {sig.name} (0 args given, expected {len(sig.args)})",
    -            stmt_expr,
    +        copy_args.append(
    +            # --> args evaluate here <--
    +            make_setter(tmp_args_buf, args_as_tuple, "memory", pos)
             )
     
    -    # Jump to function label.
    -    jump_to_func = [
    -        ["add", ["pc"], 6],  # set callback pointer.
    -        ["goto", f"priv_{sig.method_id}"],
    -        ["jumpdest"],
    -    ]
    -
    -    # Pop return values.
    -    returner = [0]
    -    if sig.output_type:
    -        output_placeholder, returner, output_size = _call_make_placeholder(stmt_expr, context, sig)
    -        if output_size > 0:
    -            dynamic_offsets = []
    -            if isinstance(sig.output_type, (BaseType, ListType)):
    -                pop_return_values = [
    -                    ["mstore", ["add", output_placeholder, pos], "pass"]
    -                    for pos in range(0, output_size, 32)
    -                ]
    -            elif isinstance(sig.output_type, ByteArrayLike):
    -                dynamic_offsets = [(0, sig.output_type)]
    -                pop_return_values = [
    -                    ["pop", "pass"],
    -                ]
    -            elif isinstance(sig.output_type, TupleLike):
    -                static_offset = 0
    -                pop_return_values = []
    -                for name, typ in sig.output_type.tuple_items():
    -                    if isinstance(typ, ByteArrayLike):
    -                        pop_return_values.append(
    -                            ["mstore", ["add", output_placeholder, static_offset], "pass"]
    -                        )
    -                        dynamic_offsets.append(
    -                            (["mload", ["add", output_placeholder, static_offset]], name)
    -                        )
    -                        static_offset += 32
    -                    else:
    -                        member_output_size = get_size_of_type(typ) * 32
    -                        pop_return_values.extend(
    -                            [
    -                                ["mstore", ["add", output_placeholder, pos], "pass"]
    -                                for pos in range(
    -                                    static_offset, static_offset + member_output_size, 32
    -                                )
    -                            ]
    -                        )
    -                        static_offset += member_output_size
    +        copy_args.append(make_setter(args_dst, tmp_args_buf, "memory", pos))
     
    -            # append dynamic unpacker.
    -            dyn_idx = 0
    -            for in_memory_offset, _out_type in dynamic_offsets:
    -                ident = f"{stmt_expr.lineno}_{stmt_expr.col_offset}_arg_{dyn_idx}"
    -                dyn_idx += 1
    -                start_label = "dyn_unpack_start_" + ident
    -                end_label = "dyn_unpack_end_" + ident
    -                i_placeholder = context.new_internal_variable(typ=BaseType("uint256"))
    -                begin_pos = ["add", output_placeholder, in_memory_offset]
    -                # loop until length.
    -                o = LLLnode.from_list(
    -                    [
    -                        "seq_unchecked",
    -                        ["mstore", begin_pos, "pass"],  # get len
    -                        ["mstore", i_placeholder, 0],
    -                        ["label", start_label],
    -                        [  # break
    -                            "if",
    -                            ["ge", ["mload", i_placeholder], ["ceil32", ["mload", begin_pos]]],
    -                            ["goto", end_label],
    -                        ],
    -                        [  # pop into correct memory slot.
    -                            "mstore",
    -                            ["add", ["add", begin_pos, 32], ["mload", i_placeholder]],
    -                            "pass",
    -                        ],
    -                        # increment i
    -                        ["mstore", i_placeholder, ["add", 32, ["mload", i_placeholder]]],
    -                        ["goto", start_label],
    -                        ["label", end_label],
    -                    ],
    -                    typ=None,
    -                    annotation="dynamic unpacker",
    -                    pos=getpos(stmt_expr),
    -                )
    -                pop_return_values.append(o)
    -
    -    call_body = list(
    -        itertools.chain(
    -            ["seq_unchecked"],
    -            pre_init,
    -            push_local_vars,
    -            push_args,
    -            jump_to_func,
    -            pop_return_values,
    -            pop_local_vars,
    -            [returner],
    -        )
    -    )
    -    # If we have no return, we need to pop off
    -    pop_returner_call_body = ["pop", call_body] if sig.output_type is None else call_body
    +    else:
    +        copy_args = make_setter(args_dst, args_as_tuple, "memory", pos)
    +
    +    call_sequence = [
    +        "seq",
    +        copy_args,
    +        [
    +            "goto",
    +            sig.internal_function_label,
    +            return_buffer,  # pass return buffer to subroutine
    +            push_label_to_stack(return_label),  # pass return label to subroutine
    +        ],
    +        ["label", return_label],
    +        return_buffer,  # push return buffer location to stack
    +    ]
     
         o = LLLnode.from_list(
    -        pop_returner_call_body,
    -        typ=sig.output_type,
    +        call_sequence,
    +        typ=sig.return_type,
             location="memory",
    -        pos=getpos(stmt_expr),
    -        annotation=f"Internal Call: {method_name}",
    +        pos=pos,
    +        annotation=stmt_expr.get("node_source_code"),
             add_gas_estimate=sig.gas,
         )
    -    o.gas += sig.gas
    +    o.is_self_call = True
         return o
    
  • vyper/old_codegen/stmt.py+21 150 modified
    @@ -1,4 +1,5 @@
     import vyper.old_codegen.events as events
    +import vyper.utils as util
     from vyper import ast as vy_ast
     from vyper.builtin_functions import STMT_DISPATCH_TABLE
     from vyper.exceptions import StructureException, TypeCheckFailure
    @@ -8,24 +9,17 @@
     from vyper.old_codegen.parser_utils import (
         LLLnode,
         getpos,
    -    make_byte_array_copier,
         make_setter,
         unwrap_location,
    -    zero_pad,
     )
    -from vyper.old_codegen.return_ import gen_tuple_return, make_return_stmt
    +from vyper.old_codegen.return_ import make_return_stmt
     from vyper.old_codegen.types import (
         BaseType,
    -    ByteArrayLike,
         ByteArrayType,
         ListType,
    -    NodeType,
    -    StructType,
    -    TupleType,
         get_size_of_type,
         parse_type,
     )
    -from vyper.utils import SizeLimits, bytes_to_int, fourbytes_to_int, keccak256
     
     
     class Stmt:
    @@ -76,20 +70,26 @@ def parse_AnnAssign(self):
             # If bytes[32] to bytes32 assignment rewrite sub as bytes32.
             if is_literal_bytes32_assign:
                 sub = LLLnode(
    -                bytes_to_int(self.stmt.value.s), typ=BaseType("bytes32"), pos=getpos(self.stmt),
    +                util.bytes_to_int(self.stmt.value.s),
    +                typ=BaseType("bytes32"),
    +                pos=getpos(self.stmt),
                 )
     
             variable_loc = LLLnode.from_list(pos, typ=typ, location="memory", pos=getpos(self.stmt),)
    +
             lll_node = make_setter(variable_loc, sub, "memory", pos=getpos(self.stmt))
    +        lll_node.annotation = self.stmt.get("node_source_code")
     
             return lll_node
     
         def parse_Assign(self):
             # Assignment (e.g. x[4] = y)
             sub = Expr(self.stmt.value, self.context).lll_node
             target = self._get_target(self.stmt.target)
    +
             lll_node = make_setter(target, sub, target.location, pos=getpos(self.stmt))
             lll_node.pos = getpos(self.stmt)
    +        lll_node.annotation = self.stmt.get("node_source_code")
             return lll_node
     
         def parse_If(self):
    @@ -136,29 +136,33 @@ def parse_Call(self):
             if isinstance(self.stmt.func, vy_ast.Name):
                 funcname = self.stmt.func.id
                 return STMT_DISPATCH_TABLE[funcname].build_LLL(self.stmt, self.context)
    +
             elif is_self_function:
    -            return self_call.make_call(self.stmt, self.context)
    +            return self_call.lll_for_self_call(self.stmt, self.context)
             else:
    -            return external_call.make_external_call(self.stmt, self.context)
    +            return external_call.lll_for_external_call(self.stmt, self.context)
     
         def _assert_reason(self, test_expr, msg):
             if isinstance(msg, vy_ast.Name) and msg.id == "UNREACHABLE":
                 return LLLnode.from_list(["assert_unreachable", test_expr], typ=None, pos=getpos(msg))
     
             reason_str_type = ByteArrayType(len(msg.value.strip()))
     
    +        # abi encode the reason string
             sig_placeholder = self.context.new_internal_variable(BaseType(32))
    +        # offset of bytes in (bytes,)
             arg_placeholder = self.context.new_internal_variable(BaseType(32))
             placeholder_bytes = Expr(msg, self.context).lll_node
     
    -        method_id = fourbytes_to_int(keccak256(b"Error(string)")[:4])
    +        method_id = util.abi_method_id("Error(string)")
     
    +        # abi encode method_id + bytestring
             revert_seq = [
                 "seq",
                 ["mstore", sig_placeholder, method_id],
                 ["mstore", arg_placeholder, 32],
                 placeholder_bytes,
    -            ["revert", sig_placeholder + 28, int(4 + get_size_of_type(reason_str_type) * 32)],
    +            ["revert", sig_placeholder + 28, int(32 + 4 + get_size_of_type(reason_str_type) * 32)],
             ]
             if test_expr:
                 lll_node = ["if", ["iszero", test_expr], revert_seq]
    @@ -388,143 +392,10 @@ def parse_Break(self):
             return LLLnode.from_list("break", typ=None, pos=getpos(self.stmt))
     
         def parse_Return(self):
    -        if self.context.return_type is None:
    -            if self.stmt.value:
    -                return
    -            return LLLnode.from_list(
    -                make_return_stmt(self.stmt, self.context, 0, 0),
    -                typ=None,
    -                pos=getpos(self.stmt),
    -                valency=0,
    -            )
    -
    -        sub = Expr(self.stmt.value, self.context).lll_node
    -
    -        # Returning a value (most common case)
    -        if isinstance(sub.typ, BaseType):
    -            sub = unwrap_location(sub)
    -
    -            if sub.typ.is_literal and (
    -                self.context.return_type.typ == sub.typ
    -                or "int" in self.context.return_type.typ
    -                and "int" in sub.typ.typ
    -            ):  # noqa: E501
    -                if SizeLimits.in_bounds(self.context.return_type.typ, sub.value):
    -                    return LLLnode.from_list(
    -                        [
    -                            "seq",
    -                            ["mstore", 0, sub],
    -                            make_return_stmt(self.stmt, self.context, 0, 32),
    -                        ],
    -                        typ=None,
    -                        pos=getpos(self.stmt),
    -                        valency=0,
    -                    )
    -            elif isinstance(sub.typ, BaseType):
    -                return LLLnode.from_list(
    -                    ["seq", ["mstore", 0, sub], make_return_stmt(self.stmt, self.context, 0, 32)],
    -                    typ=None,
    -                    pos=getpos(self.stmt),
    -                    valency=0,
    -                )
    -            return
    -        # Returning a byte array
    -        elif isinstance(sub.typ, ByteArrayLike):
    -            if not sub.typ.eq_base(self.context.return_type):
    -                return
    -            if sub.typ.maxlen > self.context.return_type.maxlen:
    -                return
    -
    -            # loop memory has to be allocated first.
    -            loop_memory_position = self.context.new_internal_variable(typ=BaseType("uint256"))
    -            # len & bytez placeholder have to be declared after each other at all times.
    -            len_placeholder = self.context.new_internal_variable(BaseType("uint256"))
    -            bytez_placeholder = self.context.new_internal_variable(sub.typ)
    -
    -            if sub.location in ("storage", "memory"):
    -                return LLLnode.from_list(
    -                    [
    -                        "seq",
    -                        make_byte_array_copier(
    -                            LLLnode(bytez_placeholder, location="memory", typ=sub.typ),
    -                            sub,
    -                            pos=getpos(self.stmt),
    -                        ),
    -                        zero_pad(bytez_placeholder),
    -                        ["mstore", len_placeholder, 32],
    -                        make_return_stmt(
    -                            self.stmt,
    -                            self.context,
    -                            len_placeholder,
    -                            ["ceil32", ["add", ["mload", bytez_placeholder], 64]],
    -                            loop_memory_position=loop_memory_position,
    -                        ),
    -                    ],
    -                    typ=None,
    -                    pos=getpos(self.stmt),
    -                    valency=0,
    -                )
    -            return
    -
    -        elif isinstance(sub.typ, ListType):
    -            loop_memory_position = self.context.new_internal_variable(typ=BaseType("uint256"))
    -            if sub.location == "memory" and sub.value != "multi":
    -                return LLLnode.from_list(
    -                    make_return_stmt(
    -                        self.stmt,
    -                        self.context,
    -                        sub,
    -                        get_size_of_type(self.context.return_type) * 32,
    -                        loop_memory_position=loop_memory_position,
    -                    ),
    -                    typ=None,
    -                    pos=getpos(self.stmt),
    -                    valency=0,
    -                )
    -            else:
    -                new_sub = LLLnode.from_list(
    -                    self.context.new_internal_variable(self.context.return_type),
    -                    typ=self.context.return_type,
    -                    location="memory",
    -                )
    -                setter = make_setter(new_sub, sub, "memory", pos=getpos(self.stmt))
    -                return LLLnode.from_list(
    -                    [
    -                        "seq",
    -                        setter,
    -                        make_return_stmt(
    -                            self.stmt,
    -                            self.context,
    -                            new_sub,
    -                            get_size_of_type(self.context.return_type) * 32,
    -                            loop_memory_position=loop_memory_position,
    -                        ),
    -                    ],
    -                    typ=None,
    -                    pos=getpos(self.stmt),
    -                )
    -
    -        # Returning a struct
    -        elif isinstance(sub.typ, StructType):
    -            retty = self.context.return_type
    -            if isinstance(retty, StructType) and retty.name == sub.typ.name:
    -                return gen_tuple_return(self.stmt, self.context, sub)
    -
    -        # Returning a tuple.
    -        elif isinstance(sub.typ, TupleType):
    -            if not isinstance(self.context.return_type, TupleType):
    -                return
    -
    -            if len(self.context.return_type.members) != len(sub.typ.members):
    -                return
    -
    -            # check return type matches, sub type.
    -            for i, ret_x in enumerate(self.context.return_type.members):
    -                s_member = sub.typ.members[i]
    -                sub_type = s_member if isinstance(s_member, NodeType) else s_member.typ
    -                if type(sub_type) is not type(ret_x):
    -                    return
    -            return gen_tuple_return(self.stmt, self.context, sub)
    +        lll_val = None
    +        if self.stmt.value is not None:
    +            lll_val = Expr(self.stmt.value, self.context).lll_node
    +        return make_return_stmt(lll_val, self.stmt, self.context)
     
         def _get_target(self, target):
             if isinstance(target, vy_ast.Name) and target.id in self.context.forvars:
    
  • vyper/old_codegen/types/check.py+2 2 modified
    @@ -6,6 +6,6 @@
     
     # Check assignment from rhs to lhs.
     # For now use make_setter for its typechecking side effects
    -def check_assign(lhs, rhs, pos, in_function_call=False):
    -    make_setter(lhs, rhs, location="memory", pos=pos, in_function_call=in_function_call)
    +def check_assign(lhs, rhs, pos):
    +    make_setter(lhs, rhs, location="memory", pos=pos)
         # TODO Refactor into an actual type-checking function
    
  • vyper/old_codegen/types/types.py+5 32 modified
    @@ -181,6 +181,7 @@ def canonicalize_type(t, is_indexed=False):
         raise InvalidType(f"Invalid or unsupported type: {repr(t)}")
     
     
    +# TODO location is unused
     def make_struct_type(name, location, sigs, members, custom_structs):
         o = OrderedDict()
     
    @@ -196,7 +197,9 @@ def make_struct_type(name, location, sigs, members, custom_structs):
     
     # Parses an expression representing a type. Annotation refers to whether
     # the type is to be located in memory or storage
    -def parse_type(item, location, sigs=None, custom_structs=None):
    +# TODO: location is unused
    +# TODO: rename me to "lll_type_from_annotation"
    +def parse_type(item, location=None, sigs=None, custom_structs=None):
         # Base and custom types, e.g. num
         if isinstance(item, vy_ast.Name):
             if item.id in BASE_TYPES:
    @@ -271,7 +274,7 @@ def parse_type(item, location, sigs=None, custom_structs=None):
     
     # byte array overhead, in words. (it should really be 1, but there are
     # some places in our calling convention where the layout expects 2)
    -BYTE_ARRAY_OVERHEAD = 2
    +BYTE_ARRAY_OVERHEAD = 1
     
     
     # Gets the maximum number of memory or storage keys needed to ABI-encode
    @@ -304,36 +307,6 @@ def get_type_for_exact_size(n_bytes):
         return ByteArrayType(n_bytes - 32 * BYTE_ARRAY_OVERHEAD)
     
     
    -# amount of space a type takes in the static section of its ABI encoding
    -def get_static_size_of_type(typ):
    -    if isinstance(typ, BaseType):
    -        return 1
    -    elif isinstance(typ, ByteArrayLike):
    -        return 1
    -    elif isinstance(typ, ListType):
    -        return get_size_of_type(typ.subtype) * typ.count
    -    elif isinstance(typ, MappingType):
    -        raise InvalidType("Maps are not supported for function arguments or outputs.")
    -    elif isinstance(typ, TupleLike):
    -        return sum([get_size_of_type(v) for v in typ.tuple_members()])
    -    else:
    -        raise InvalidType(f"Can not get size of type, Unexpected type: {repr(typ)}")
    -
    -
    -# could be rewritten as get_static_size_of_type == get_size_of_type?
    -def has_dynamic_data(typ):
    -    if isinstance(typ, BaseType):
    -        return False
    -    elif isinstance(typ, ByteArrayLike):
    -        return True
    -    elif isinstance(typ, ListType):
    -        return has_dynamic_data(typ.subtype)
    -    elif isinstance(typ, TupleLike):
    -        return any([has_dynamic_data(v) for v in typ.tuple_members()])
    -    else:
    -        raise InvalidType(f"Unexpected type: {repr(typ)}")
    -
    -
     def get_type(input):
         if not hasattr(input, "typ"):
             typ, len = "num_literal", 32
    
  • vyper/semantics/namespace.py+2 2 modified
    @@ -22,7 +22,7 @@ class Namespace(dict):
         def __init__(self):
             super().__init__()
             self._scopes = []
    -        # FLAG cyclic imports!
    +        # NOTE cyclic imports!
             from vyper.builtin_functions.functions import get_builtin_functions
             from vyper.semantics import environment
             from vyper.semantics.types import get_types
    @@ -62,7 +62,7 @@ def enter_scope(self):
             Called as a context manager, e.g. `with namespace.enter_scope():`
             All items that are added within the context are removed upon exit.
             """
    -        # FLAG cyclic imports!
    +        # NOTE cyclic imports!
             from vyper.semantics import environment
     
             self._scopes.append(set())
    
  • vyper/semantics/types/function.py+23 12 modified
    @@ -96,6 +96,7 @@ def __init__(
             self,
             name: str,
             arguments: OrderedDict,
    +        # TODO rename to something like positional_args, keyword_args
             min_arg_count: int,
             max_arg_count: int,
             return_type: Optional[BaseTypeDefinition],
    @@ -321,13 +322,6 @@ def from_FunctionDef(
                 type_definition = get_type_from_annotation(
                     arg.annotation, location=DataLocation.CALLDATA, is_immutable=True
                 )
    -            if isinstance(type_definition, StructDefinition) and type_definition.is_dynamic_size:
    -                # this is a temporary restriction and should be removed once support for dynamically
    -                # sized structs is implemented - https://github.com/vyperlang/vyper/issues/2190
    -                raise ArgumentException(
    -                    "Struct with dynamically sized data cannot be used as a function input", arg
    -                )
    -
                 if value is not None:
                     if not check_constant(value):
                         raise StateAccessViolation(
    @@ -367,7 +361,7 @@ def from_AnnAssign(cls, node: vy_ast.AnnAssign) -> "ContractFunction":
             """
             Generate a `ContractFunction` object from an `AnnAssign` node.
     
    -        Used to create function definitions for public variables.
    +        Used to create getter functions for public variables.
     
             Arguments
             ---------
    @@ -402,7 +396,7 @@ def method_ids(self) -> Dict[str, int]:
     
             * For functions without default arguments the dict contains one item.
             * For functions with default arguments, there is one key for each
    -          function signfature.
    +          function signature.
             """
             arg_types = [i.canonical_type for i in self.arguments.values()]
     
    @@ -414,6 +408,20 @@ def method_ids(self) -> Dict[str, int]:
                 method_ids.update(_generate_method_id(self.name, arg_types[:i]))
             return method_ids
     
    +    # for caller-fills-args calling convention
    +    def get_args_buffer_offset(self) -> int:
    +        """
    +        Get the location of the args buffer in the function frame (caller sets)
    +        """
    +        return 0
    +
    +    # TODO is this needed?
    +    def get_args_buffer_len(self) -> int:
    +        """
    +        Get the length of the argument buffer in the function frame
    +        """
    +        return sum(arg_t.size_in_bytes() for arg_t in self.arguments.values())
    +
         @property
         def is_constructor(self) -> bool:
             return self.name == "__init__"
    @@ -473,10 +481,8 @@ def to_abi_dict(self) -> List[Dict]:
             typ = self.return_type
             if typ is None:
                 abi_dict["outputs"] = []
    -        elif isinstance(typ, TupleDefinition):
    +        elif isinstance(typ, TupleDefinition) and len(typ.value_type) > 1:  # type: ignore
                 abi_dict["outputs"] = [_generate_abi_type(i) for i in typ.value_type]  # type: ignore
    -        elif isinstance(typ, StructDefinition):
    -            abi_dict["outputs"] = [_generate_abi_type(v, k) for k, v in typ.members.items()]
             else:
                 abi_dict["outputs"] = [_generate_abi_type(typ)]
     
    @@ -498,6 +504,11 @@ def _generate_abi_type(type_definition, name=""):
                 "type": "tuple",
                 "components": [_generate_abi_type(v, k) for k, v in type_definition.members.items()],
             }
    +    if isinstance(type_definition, TupleDefinition):
    +        return {
    +            "type": "tuple",
    +            "components": [_generate_abi_type(i) for i in type_definition.value_type],
    +        }
         return {"name": name, "type": type_definition.canonical_type}
     
     
    
  • vyper/semantics/types/indexable/sequence.py+2 0 modified
    @@ -116,7 +116,9 @@ def __init__(self, value_type: Tuple[BaseTypeDefinition, ...]) -> None:
             # always use the most restrictive location re: modification
             location = sorted((i.location for i in value_type), key=lambda k: k.value)[-1]
             is_immutable = next((True for i in value_type if getattr(i, "is_immutable", None)), False)
    +
             super().__init__(
    +            # TODO fix the typing on value_type
                 value_type,  # type: ignore
                 len(value_type),
                 f"{value_type}",
    
  • vyper/semantics/types/value/array_value.py+2 3 modified
    @@ -1,9 +1,9 @@
    -import math
     from typing import Type
     
     from vyper import ast as vy_ast
     from vyper.exceptions import CompilerPanic, StructureException, UnexpectedValue
     from vyper.semantics import validation
    +from vyper.utils import ceil32
     
     from ..abstract import ArrayValueAbstractType, BytesAbstractType
     from ..bases import BasePrimitive, DataLocation, ValueTypeDefinition
    @@ -61,8 +61,7 @@ def size_in_bytes(self):
             # because this data type is single-bytes, we make it so it takes the max 32 byte
             # boundary as it's size, instead of giving it a size that is not cleanly divisble by 32
     
    -        # TODO adding 64 here instead of 32 to be compatible with parser - fix this!
    -        return 64 + math.ceil(self.length / 32) * 32
    +        return 32 + ceil32(self.length)
     
         @property
         def canonical_type(self) -> str:
    
  • vyper/utils.py+29 1 modified
    @@ -1,5 +1,7 @@
     import binascii
     import functools
    +import sys
    +import traceback
     from typing import Dict, List, Union
     
     from vyper.exceptions import InvalidLiteral
    @@ -13,18 +15,37 @@
     
         keccak256 = lambda x: _sha3.sha3_256(x).digest()  # noqa: E731
     
    +try:
    +    # available py3.8+
    +    from functools import cached_property
    +except ImportError:
    +    from cached_property import cached_property  # type: ignore
    +
     
     # Converts four bytes to an integer
     def fourbytes_to_int(inp):
         return (inp[0] << 24) + (inp[1] << 16) + (inp[2] << 8) + inp[3]
     
     
    +# utility function for debugging purposes
    +def trace(n=5, out=sys.stderr):
    +    print("BEGIN TRACE", file=out)
    +    for x in list(traceback.format_stack())[-n:]:
    +        print(x.strip(), file=out)
    +    print("END TRACE", file=out)
    +
    +
     # converts a signature like Func(bool,uint256,address) to its 4 byte method ID
     # TODO replace manual calculations in codebase with this
     def abi_method_id(method_sig):
         return fourbytes_to_int(keccak256(bytes(method_sig, "utf-8"))[:4])
     
     
    +# map a string to only-alphanumeric chars
    +def mkalphanum(s):
    +    return "".join([c if c.isalnum() else "_" for c in s])
    +
    +
     # Converts string to bytes
     def string_to_bytes(str):
         bytez = b""
    @@ -77,6 +98,8 @@ def calc_mem_gas(memsize):
     # Specific gas usage
     GAS_IDENTITY = 15
     GAS_IDENTITYWORD = 3
    +GAS_CODECOPY_WORD = 3
    +GAS_CALLDATACOPY_WORD = 3
     
     # A decimal value can store multiples of 1/DECIMAL_DIVISOR
     MAX_DECIMAL_PLACES = 10
    @@ -139,7 +162,7 @@ def in_bounds(cls, type_str, value):
         "send",
     }
     
    -# List of valid LLL macros. Used for colorising LLL output
    +# List of valid LLL macros.
     VALID_LLL_MACROS = {
         "assert",
         "break",
    @@ -299,3 +322,8 @@ def annotate_source_code(
         cleanup_lines += [""] * (num_lines - len(cleanup_lines))
     
         return "\n".join(cleanup_lines)
    +
    +
    +__all__ = [
    +    "cached_property",
    +]
    

Vulnerability mechanics

Generated on May 9, 2026. Inputs: CWE entries + fix-commit diffs from this CVE's patches. Citations validated against bundle.

References

6

News mentions

0

No linked articles in our index yet.