At work in our AI discussion forum someone posted this video:
https://www.youtube.com/watch?v=v4F1gFy-hqgOne thing I'm wrestling with is... how do devs/LLM-wranglers ensure we get good "value for money" in terms of the pre-prompting stuff? To a large extent, decent coding practices are built into LLM. (Admittedly, the LLMs have been more trained on toy example which might cause them to be more likely to take shortcuts.) So in theory, it might not make sense to stuff your context window with things the LLM already knows deeply. (Or maybe this is a false economy, and that the number of tokens spent compared to what can be saved is small... this is why non-deterministic programming might always have an aspect "as much an art as a science.")
Another topic I want to refine my thinking on is testing. The TDD (Test Driven Development) mindset makes a case for itself more strongly in this LLM world in versus what we had before - in part because of the "pain in the butt factor" of it all. In UI at least, most medium to low level tests depend on content pieces (page structure, test-ids) that aren't in place UNTIL implementation - real chicken and egg stuff, where you would have to architect fine grained structure while writing the test, which isn't really a comfortable way to work. But with an AI, that discomfort might not exist (or at least not felt by humans)
Finally I wonder about the "software fundamental" of unit test. Like, I LOVE that this video suggests "grey box" modularization - limiting coupling to specific points between higher level modules. To me that points to where the real value in testing is, at integration of those modules, and end to end testing. In the old world, it was difficult to get real value from unit tests, tests that were real canaries in the coal mine vs false alarms, so I was always in the camp that preferred higher level tests.
But I'm not quite sure what the answer is for the LLM world. Anecdotes suggest that sometimes LLM care about tests too much, will keep code around just because there's a test for it after a refactoring. I appreciate that at the very least LLMs have helped lower the cost of unit test creation.
