November 6, 2023 Update
We have currently switched the bundler service from Stackup to another service provider. When you are certain that there is no issue with your contract but the bundler fails to estimate or submit transactions properly, you should promptly switch to another service provider.

Introduction

Bundler is an essential infrastructure in ERC4337 that bundles transactions from AA wallet users for on-chain submission. All operations of AA accounts are submitted through the bundler (currently, you can directly interact with the account contract).

When integrating the bundler into a wallet application, various peculiar issues may arise, such as AA23 reverted (or OOG) or AA13 initCode failed or OOG. I encountered several challenges during the development of Bitizen Wallet, especially in Paymaster development, and I hope that sharing my experiences can be helpful to others.

Basics and concepts

First, let's familiarize ourselves with some rules:

1. nonce The nonce of an account contract is incremented under the management of the EntryPoint. With each transaction bundled and submitted to the blockchain, the nonce must be increased by 1.
2. initCode If an account contract has been deployed, the initCode in the next userOperation must be empty.
3. paymasterAndData The data format is "paymaster address + other data," where the additional data can be decoded and utilized during the validatePaymasterUserOp phase within the Paymaster.

Special Notice Regarding Paymaster: Before using the Paymaster, it needs to be deposited and staked in the EntryPoint. The amount to be staked depends on the requirements of the integrated bundler service and can currently be set to 1 wei. The deposited amount will be used to cover the transaction fees of the account contract, while the staked amount will serve as a locked collateral deposit for reputation assurance (which can be unlocked and withdrawn).

How To

Assuming that our account contract, account factory contract, and paymaster contract have all been developed, and we have entered the phase of integrating them into the wallet application. The following issues we encounter will be related solely to the wallet application's call to the bundler.

There's no need to be surprised when encountering issues, whether it's:

• AA23 reverted (or OOG)
• AA13 initCode failed or OOG
• Or revert without any error message

First, we obtain the UserOperation that was sent when making the request to the eth_estimateUserOperationGas method of the bundler.

{
 "jsonrpc": "2.0",
 "id": 1,
 "method": "eth_sendUserOperation",
 "params": [{
   "sender": "0x1500c333bF4E25265093c272BE0611Af4c76585a",
   "nonce": "0x0",
   "initCode": "0x6da106206e6a92d317ab147a888c7d97644d878e784a7998000000000000000000000000dad2628b48c4a2a862d216926b374c08581571d50000000000000000000000001500c333bf4e25265093c272be0611af4c76585a0000000000000000000000000000000000000000000000000000000000000000",
   "callData": "0xb61d27f60000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000000",
   "callGasLimit": "0x814C",
   "verificationGasLimit": "0x5DDAA",
   "preVerificationGas": "0xB53C",
   "maxFeePerGas": "0x12A05F200",
   "maxPriorityFeePerGas": "0x12A05F200",
   "paymasterAndData": "0xe8EF01c17E76D7f365543AA0b4f1190d514Ecd04000000000000000000000000c645902921501e9ee94abc8bf945d5bea6f340f900000000000000000000000000000000000000000000000000000000000001f400000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000000",
   "signature": "0xdfbd241c98f5aa17c35852c556dfdc8d27995a2bc820c18630e02410f6d5453d550ce58f3e74ff4437cb2a09ea258d427f7e85400cef39734eb5c4411067f33e1b"
  },
  "0x5FF137D4b0FDCD49DcA30c7CF57E578a026d2789"
 ]
}

This is the JSON representing the UserOperation that we passed to the Bundler. Next, we will utilize Foundry, a smart contract development toolchain. We will convert this UserOperation into Solidity code and use Foundry to simulate its execution in order to locate the issue.

// SPDX-License-Identifier: Unlicense
pragma solidity ^0.8.13;

import "forge-std/Script.sol";
import "account-abstraction/core/EntryPoint.sol";

contract TestPaymasterScript is Script {
    function run() external {
        UserOperation memory userOp = UserOperation({
            sender: 0x1500c333bF4E25265093c272BE0611Af4c76585a,
            nonce: 0x0,
            initCode: hex"6da106206e6a92d317ab147a888c7d97644d878e784a7998000000000000000000000000dad2628b48c4a2a862d216926b374c08581571d50000000000000000000000001500c333bf4e25265093c272be0611af4c76585a0000000000000000000000000000000000000000000000000000000000000000",
            callData: hex"b61d27f60000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000000",
            paymasterAndData: hex"e8EF01c17E76D7f365543AA0b4f1190d514Ecd04000000000000000000000000c645902921501e9ee94abc8bf945d5bea6f340f900000000000000000000000000000000000000000000000000000000000001f400000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000000",
            signature: hex"16b15f88bbd2e0a22d1d0084b8b7080f2003ea83eab1a00f80d8c18446c9c1b6224f17aa09eaf167717ca4f355bb6dc94356e037edf3adf6735a86fc3741f5231b",
            callGasLimit: 0x814C,
            verificationGasLimit: 0x5DDAA,
            preVerificationGas: 0xB53C,
            maxPriorityFeePerGas: 0x12A05F200,
            maxFeePerGas: 0x12A05F200
        });
        EntryPoint(payable(0x5FF137D4b0FDCD49DcA30c7CF57E578a026d2789)).simulateHandleOp(userOp, address(0), "");
    }
}

Please place this file in the script/TestPaymaster.s.sol location of your Foundry project. Then execute forge script script/TestPaymaster.s.sol --fork-url $POLYGON_RPC_URL -vvvv. This will provide you with detailed execution information for each step to help you locate the issue.

In the issues you have encountered, most of them are related to insufficient gas limits, which can result in failures during various stages of a transaction. For example, when using a paymaster to pay transaction fees with ERC20 tokens within a transaction, it is common to face situations where you have approved the token for the paymaster in the calldata of your account, but when executing the paymaster's postOp function, you discover that the allowance is set to zero.

At this point, you need to carefully examine the detailed logs provided by Foundry to identify any issues that occurred during the approval stage.

Unexpected

However, unexpected things can happen, such as the integration of the Stackup Bundler.

The working mechanism of our ERC20 Token Paymaster involves Account execute expected operations after going through the validateUserOp of the Account and the validateUserOp of the paymaster. This includes approve ERC20 tokens to the paymaster for paying miner fees. But there was an issue with the stackup bundler where, during call the paymaster's postOp, the query result indicated that the Account was not approved to use the tokens.

By debugging their bundler, it was discovered that the issue might be caused by the bundler setting the callGasLimit too low, resulting in the Account's operations not being executed successfully.

We implemented a solution to trust the calls where tx.origin == address(0) and not deduct tokens to ensure successful gas estimation. This allows the user to make the actual call and deduct tokens in the real transaction.

Conclusion

For any UserOperation, we recommend setting the callGasLimit and verificationGasLimit to 10,000,000 when initially calling the eth_estimateUserOperationGas method on the bundler. Additionally, set the maxFeePerGas and maxPriorityFeePerGas to 1 (in wei, not gwei). There is no need for special settings for preVerificationGas.

These recommendations are made to ensure that the validate and account contract execution operations, as well as the paymaster's postOp operation, have sufficient gas to execute and allow for a successful estimation.

After successfully estimating gas, there is still a possibility that the bundler rejects the eth_sendUserOperation call due to the preVerificationGas not meeting its requirements. That's why we mentioned adjusting callGasLimit, verificationGasLimit, and gasPrice only during the initial gas estimation. This is because we need to use the actual values to calculate the precise preVerificationGas for the bundler.

If you encounter any issues, you can refer to the "How To" section and make the most of Foundry. 😄