hidden

it is end of 2024, I recently re-discovered my notes from may 2022, lets see how many of them became true.

[x] AI search engine
[x] AI’s good with tools not precise (recurse it)
[ ] ai package installer
[ ] obsolete code/file/command update (e.g. old hexo to new format, old dependencies to new)
[x] gpt twitch chat interaction
[ ] how to train llms to delete?

hidden

About a week ago, I have listened to a Lex Fridman Podcast with Charan Ranganath and took notes on it.

I immediately thought it could be applied in LLMs, as many ideas that C.R. proposed are quite interesting, such as:

• Simillar neurons are used by the brain when predicting the future and remembering the past
• Humans don’t replay the past, we imagine what the past could’ve been by taking bits and pieces
• “You don’w want to remember more, but better” (Remembering things at a higher abstract, cramming vs actually knowing)
  • Maybe there’s something here
• He suggested Internal Models of Events
• Forgetting and Retrieval failure

Internal Models of Events

Internal Models of Events are formed with both Semantic and Episodic memory at particular points of high prediction error. And those points are when its maximally optimal to encode as episodic memory.

Thoughts

• wait do human store episodic memories for training while sleeping?