Digit recognition project with GPT-4
Yesterday I built a crappy handwritten digit recognition app with GPT-4. FYI, it’s here.
I’ll share my takeaways here. I’ll be stating my thoughts as facts for brevity, but it’s all just my opinion based on my use cases.
So, here they go, just a brain dump:
- I did a ~30 hour long project with GPT-3 earlier, and now this one with GPT-4. The diff is smaller than what I expected based on the new wave of hype. GPT-4 is significantly better than GPT-3, but I’d say it’s an incremental improvement rather than a game-changer. (GPT-3 was a game-changer!)
- The ~10,000-character limit is a very strong limitation on the web UI. API access with those huge token counts might unlock significant new uses.
- In code generation, GPT-3 was ~92% correct, GPT-4 is ~98% correct. But I still needed to understand 100% of the code to fix up that 2% of buggy code. The creator of the Conway-Pong game was lucky to get a 100% working code right away.
- Probably because of the load people put on it, ChatGPT is on the verge of being useless. It didn’t load the chat history at all, encountered a network error in 60% of its responses, and was very slow overall.
- I can see how this is useful for generating code for greenfield projects, but for contributing to larger projects, the key will be the smart context selection that Cody and Copilot do.
- For any project when working with ChatGPT, I’ve found the following workflow very useful:
- Give some context, state your goal, and ask for the rough steps to achieve the goal.
- Ask GPT to aid you with step 1, step 2, etc.
- If needed, use more levels of this, but make sure that both you and GPT have the higher contexts in memory.
- It’s the same with people, but GPT is more powerful than a person if you use this flow because of its knowledge and speed.
My main takeaway: everyone should do a project like this and learn to use ChatGPT, and work with its limitations! This is the most powerful new tool I used in a decade.