Awesome!
Hey guys I've been spending almost 10 hours yesterday to make your scaffolding work. At first I just wanted to adapt it to drop VLLM and make it just call an already running OpenAI compatible API server instead (my local llama.cpp server), but in the end I had to adapt many things because I had an issue with for example answer happening before the LLM had a chance to have the output of the code execution. I think I overcomplicated the approach; I don't exactly remember everything I did. This was almost entirely vibe coded using local LLM like oss-20b, qwen coder, apriel and qwen 30A3B thinking.
But in the end, those 10 hours were totally worth it! I now get the expected behavior: and this is working so well! The LLM never gets stuck, and it shipped great insights on my test data examples!
I don't really know if I should make a pull request or if I should just publish my fork without doing so. Because again I am not really able to explain all the changes I made, because of the vibe coding and also because I was in a mental tunnel focused mode and I was 36hours awake when I finally made it work! And mainly because I ignore if the issue with the steps order (execution output) is present on your main branch. I guess not and I did a mess, breaking it in the first place. That said I don't really know how my changes could be related. But if you think I should, I would do so with pleasure.
Really great job again and thank you!
And mainly because I ignore if the issue with the steps order (execution output) is present on your main branch.
I just watched your demo video, and noticed the behavior regarding the placement of the execution output is indeed correct, so I broke it at some point, in my early changes!
Thank you so much for your hard work and debugging efforts. We truly appreciate your contribution and look forward to maximizing the value of your work.
If you’ve already built the GGUF model and configured the related environment, we warmly welcome you to directly upload a DeepAnalyze-8B-GGUF model under your account. It would also be great if you could add a LLAMACPP_README.md file to the DeepAnalyze GitHub repository, explaining in detail how to use DeepAnalyze with llama.cpp, and include an introduction to llama.cpp and GGUF in the main README.
We’ve received many requests from researchers and developers in the community for DeepAnalyze’s GGUF and llama.cpp support, but our own experience in this area is relatively limited. That’s why we’d be very grateful for your help in contributing to this effort.
Once again, thank you for your support and dedication.