Alex' Gardenアレックスの庭

6 min read

Debugging and speed for AI-native software (2026-04-27, 13:49)

Source voice memo. Follow-up to 2026-04-27-Agent-native software architecture (24 min later, same train of thought). Synthesized into Agent-Native Software Architecture and There is no main js.

Transcript

I think also in order to make these AI native software projects better. Currently they, they are good. They are, they have the use cases but they still suck too much.

Like speed is one thing. And I think like that speed problem, if that’s kind of solved, for example, you can use um a model that has 10 times the speed as current ones, with the same smartness, of course, that would be a big boost. Uh, and currently you can also with this clean architecture, if you follow it, choose different models for different sub- agents, so that’s also a reason why you should keep it modular that way.

Um But yeah, if you just extrapolate it further, apart from smartness and just well, just focus on the speed and cost of it.

If you would make the speed like 20 or 30 times as fast and then the cost like 20 times as cheap, that was the status quo, I think this kind of software would already be really viable also for the mainstream. Essentially because it’s just so much more flexible and well, pretty interesting and you could, with that kind of speed also render custom UI’s or anything.

But I guess for the UI part of it, you could just, well, as I said before, embed the Cloud code SDK in your application and build a front end for it that then renders out some data that was stored by the agent essentially. But well, I’m not, on that UI or usability question, I’m not really a fan of just strapping on a chat bar and then letting you interact with the agent.

There has been many talks for um this new era of software where it’s not just a UI anymore and you click on it, so but it’s more intent based and automatic.

Um there is a whole discussion going on there, it’s a different topic. It’s even higher level in abstraction, I guess. But yeah.

Well, also one thing I noticed while implementing this Agent Native software is that it has its own sort of debugging.

So, I noticed I first of all created the plan to implement the project and then I just let it rip and checked if the setup works fine and then check if the functionalities work fine. And the first run, honestly, even if you put a lot of thought in it, it’s just really slow and that’s mostly because you the skills that have been auto generated are not perfect. Like mostly when you set it up the first time, it just hallucinates some stuff or it doesn’t find the best path or some bugs are in there and it’s quite slow.

And so then this is a, well, it’s essentially agent debugging. I don’t know how to call it, but it’s not traditional debugging or performance analysis where you check which function or which database operation takes so long, it’s more like, yeah, which class or which sub-agent currently takes long and that’s usually not because the operation is so hard, it’s usually because the path it should take is not either not possible in the way it’s described or well just inefficient.

Also one big optimization technique I found was to not pass like file contents or results of steps in between agents directly but rather use files, temporary files.

And you could think of that like a variable but not in the sense of a pointer. So, as I said before, context window is the RAM of the program and you can use, you can either define new variables which take up space in the context window, just to keep it in the conversation flow or you can just use temporary files where you put out the result and then when another-when you pass it off to another sub-agent, you basically um just pass the file reference, so it doesn’t have to always read the content and that’s saving essentially output tokens because you do not have to call the next agent with all the results of the previous agent, just to say the result is in this file and that’s basically saving output tokens, but it’s also saving input tokens because the result can be read on demand and that’s, I don’t know.

It’s kind of like- it’s not exactly the same but it’s kind of like having variables and that are passed by values and variables that are just pointers. But in this kind of agent native architecture, currently at least, maybe that’s comparable to back in the day where you just had like a few kilobytes of RAM, but currently it’s just a pretty neat trick to make everything faster and cheaper that I found.

But well, then there’s other things that, for example, as I noted earlier, you could have this instruction file that describes a golden path that just does not work perfectly and so if you don’t fix that or if the agent doesn’t fix it like itself, or you have a heal agent, that it will always start with that jacked path forward and it will always be slow, but well, that’s the job of the developer, I guess to check the runs, see the logging or well, logging it- doesn’t really have logging, it just has this thought process you can see it, Cloud code.

That’s your logging essentially and then to check what’s taking long and where the agent makes unnecessary tool calls.

But well, recently it’s been getting kind of hard to debug that because Cloud code or Anthrop in general keeps making the thought process of AI tools more and more obscure.

You can’t see the thought- the thinking part of the model that well anymore and also tool calls, so that’s kind of annoying, but currently it’s still possible, of course.

Graph View

Backlinks