Generating proof with a ZKProgram throws an unknown exception without any stack trace

[waxen pagoda]

I'm having problem debugging my ZkProgram, which looks like this:

export const FinalizeRound2 = ZkProgram({
  name: 'finalize-round-2',
  publicInput: PublicInput,
  publicOutput: PublicOutput,
  methods: {
    firstStep: {
      privateInputs: [...],
      method(...) {...}
    },
    nextStep: {
      privateInputs: [...],
      method(...) {
        ...
        Provable.log('Inside-proof logic is correct!');
        return new PublicOutput(...);
      }
    },

This ZkProgram is for a recursive update, the firstStep method works well, but the nextStep throws an exception without any stack trace (other bugs I met when develop ZkApp have specific stack trace), as in the attached photo. I want to ask if anyone had came across a similar problem before as I have no clue what is causing this 🙂

Also, all the logic in my nextStep method is correct as all the asserts were passed and the final log before return was run. Apperciate your helps 😁

[rapid eagle]

Can you paste error ?

[waxen pagoda]

thrown: Array [
      0,
      Array [
        248,
        MlBytes {
          "c": "Assert_failure",
          "l": 14,
          "t": 0,
        },
        -11,
      ],
      Array [
        0,
        MlBytes {
          "c": "src/lib/pickles/step.ml",
          "l": 23,
          "t": 0,
        },
        511,
        37,
      ],
    ]

sure, here is the error

[crystal jay]

You see memory problems. Try to add cache when compiling the contract, and maybe on 3rd try it will pass. I would recommend checking memory by using this code:

class Memory {
  static rss: number = 0;
  constructor() {
    Memory.rss = 0;
  }

  public static info(description: string = ``) {
    const memoryData = process.memoryUsage();
    const formatMemoryUsage = (data: any) =>
      `${Math.round(data / 1024 / 1024)} MB`;
    const oldRSS = Memory.rss;
    Memory.rss = Math.round(memoryData.rss / 1024 / 1024);
    /*
    const memoryUsage = {
      rssDelta: `${oldRSS === 0 ? 0 : Memory.rss - oldRSS} MB`,
      rss: `${formatMemoryUsage(
        memoryData.rss
      )} -> Resident Set Size - total memory allocated`,
      heapTotal: `${formatMemoryUsage(
        memoryData.heapTotal
      )} -> total size of the allocated heap`,
      heapUsed: `${formatMemoryUsage(
        memoryData.heapUsed
      )} -> actual memory used during the execution`,
      external: `${formatMemoryUsage(
        memoryData.external
      )} -> V8 external memory`,
    };
    */

    console.log(
      `RSS memory ${description}: ${formatMemoryUsage(memoryData.rss)}${
        oldRSS === 0
          ? ``
          : `, changed by ` + (Memory.rss - oldRSS).toString() + ` MB`
      }`
    );
  }
}

[obsidian reef]

Also, all the logic in my nextStep method is correct as all the asserts were passed and the final log before return was run

This is a misunderstanding, I think. These methods are executed multiple times, so seeing a log doesn't mean that the execution reached the log line on every execution. Can you paste your whole circuit code?

[waxen pagoda]

I've already using cache and my maximum memory for nodejs is 8GB. And I believe this script should be run when inside my proving script, right?

[crystal jay]

Yes, just add Memory.info("step 1") to your script after every call to ZkProgram. Usually you see such kind of errors when RSS memory is 7GB

btw, what is your o1js version?

[waxen pagoda]

Hmm, the reason I think so is if one of the above constraints doesn't match, it will throw an unmatch assert equal error. And after I figured out all of them, this errow showed up.

Here is the link to the full code https://github.com/auxo-zk/Distributed-key-generation/blob/replace-merkle-map/src/contracts/Round2.ts#L150-L314
It's quite long so I don't think I should copy it in here 🥲

[waxen pagoda]

I'm using 0.14.1

[crystal jay]

Try also to compile small SmartContract BEFORE your ZkProgram. Don't use it, just compile

[waxen pagoda]

thanks for the insight, I will try running this

[waxen pagoda]

I'm setting proofEnable to false to avoid compiling the contracts every time running the test, could it be a problem?

[crystal jay]

You'd better test on testworld2, much closer to reality

[obsidian reef]

let actionState = updateOutOfSnark(input.previousActionState, [
          Action.toFields(input.action),
        ]);

This doesn't look right to me. Maybe you need to wrap this in a witness. That function updateOutOfSnark uses a js if statement, which you can't use in the circuit.

[waxen pagoda]

Hmm, it works without problem in my other ZkProgram, here is code of the function:

export function updateOutOfSnark(state: Field, action: Field[][]) {
  if (action === undefined) return state;
  let actionsHash = AccountUpdate.Actions.hash(action);
  return AccountUpdate.Actions.updateSequenceState(state, actionsHash);
}

[waxen pagoda]

Really, I didn't know that, I'm using only Local for testing. Is the testworld2 network using v0.14.1 too?

[waxen pagoda]

For example, this program runs normally using the same function https://github.com/auxo-zk/Distributed-key-generation/blob/replace-merkle-map/src/contracts/Round1.ts#L123-L293

[crystal jay]

Try also to remove from your ZkProgram Level1Witness completely (it is ok that the business logic will be bad) just to see what will happen. 70% probability that this error will go away.

[crystal jay]

Yes, you can call testworld using o1js

[waxen pagoda]

yeah I will try that, I believe the purpose is to reduce the constraints size right?

Update: I ran the script and it shows the program was using 2073 MB at the moment before calling the nextStep prover

[obsidian reef]

Why did you call the function updateOutOfSnark? Your intention is that the code is executed outside of the circuit, and the result is brought in? Or do you expect the logic to execute in the circuit?

AFAIK, if statements are removed from the code during circuit compilation, so it would make sense to me that the code works sometimes - the times when the if statement is false, it's the same as if you never evaluated it.

[crystal jay]

I think there are no errors in the contract; it is just too big and needs some diet.🤣

[waxen pagoda]

Sorry for the bad naming, the idea of the function is to mimic the update of action state (by hashing the previous action state with the current action hash). I will test to see if this caused the problem, thanks for pointing out 😁

[obsidian reef]

Sorry if I end up sending you down a rabbit hole for the wrong reason. To me, the error reads as an assertion failure in the circuit, that's not mimicked in the JS. Like I said, it executes multiple times. First it tries in JS, so it can raise a helpful stack trace for you. Once it passes JS, it runs the pickles proof, which spits out an error like this. One reason why it can pass in JS and fail in pickles is a memory issue, but another reason is that the JS code and the pickles proof are not the same logic. This can happen in cases where you use a dynamic array, or conditional logic that pickles doesn't understand.