This past holiday weekend, my wife and I traveled to the East Coast to visit family. Spending time with my 3-year-old niece was a highlight, and one of our experiences sparked some interesting thoughts about LLMs.

My parents got her a tricycle earlier this year, and watching her learn to ride it around the house was entertaining.

Before this weekend, she relied on others to push her around while she did some basic steering. She didn’t know how to pedal on her own yet. Simply explaining how to do it wasn’t enough – she’s only 3. Demonstrating wasn’t an option either—I’m a bit too big for her tricycle! So how did she learn? Instead of pushing her tricycle around the whole house, I gave her an initial push and encouraged her to keep going. Soon she figured out how to push the pedals correctly. Sounds a lot like prompting an LLM!

When she reached the end of the room, she didn’t know how to turn back. I told her to make a U-turn, but the problem was she had no idea what that meant. She’s never done a U-turn before! I had to guide her through making a U-turn a few times, and then she understood what to do when she ended up at the edge of the room. Sounds like giving few-shot examples to an LLM! We’ve encountered this a lot when building our chatbot at FunnelStory. LLMs, which are trained on mostly public datasets, aren’t familiar with concepts in our unique startup product/platform.

Sometimes, family members stood in her path. Instead of running into them, I told her to say “beep beep” so they’d move. A simple prompt, but effective!

She also sometimes got stuck going around the kitchen island. Initially, she didn’t realize she could go back and make wider turns. I taught her that sometimes you need to go back and try again. After a couple of tries, she mastered it, and didn’t need someone to get her unstuck.

This experience made me think about prompting, few-shot examples, and thought engineering with LLMs. Just like with kids, while LLMs might not get it right the first time, they have the potential to learn. If we assume they can do it and just need the right training or prompting, we can get more creative and find better solutions.