Forgive but don't forget


Interesting title for one of my typically technical blog posts, right? Don't worry, I'll explain.

If you knew me before I started this nerd blog of mine, you possibly saw a Facebook post about a tool I built to aid in text memorization several years ago. If not, well, I built a text memorization tool called MemorEquip several years ago. And while it can be used to help you memorize any kind of text, I built it for myself with Bible memorization in mind. I may have also built it with applications of artificial intelligence in mind though so don't think me too much of a saint. (If you're more interested in applications of AI, you might find my next post more interesting - it actually already exists and is linked at the bottom of this post).

Anyway, so "forgive" because forgiveness is rather foundational to the Christian faith, and "don't forget" because that's the point of the software. Additionally, forgive me for how bad the site was the first time around.

That said, I've attempted to make a number of improvements. Probably the biggest (yet least noticeable) change is to the organization of my code. The first time around, I let things get rather messy which made any feature additions or bug fixes a major pain so I avoided them like the plague.

But with that re-architecture behind me, I felt motivated to add some cool new stuff to the tool.


Re-design

While I'm not a user interface/experience expert by any means, I do like to think I improved a few pieces of the UI.

First, I just cleaned up the sidebar menu. I'd say it looks more modern and less Web 2.0. Sorry, I was thinking I'd do before and after shots but got lazy.

Next, I attempted a better interface for requesting hints in the actual text memorization portion of the site. I typically never use the actual buttons to request hints because 90% of the hints I request apply only to the current word (like give-me-the-next-letter). And there are shortcut keys for those. But I realize not everybody finds moving your hand a few inches to use the mouse as taxing as I do so I opted to make a cleaner clicky UI.


xkcd ui change

The buttons surround the textbox. I tried to come up with semi-symbolic depictions of the functions of each hint button to avoid large paragraphs explaining what each button does. That said, hovering over each button should give a little more plain English of an explanation as well as the shortcut key if one exists for that hint.


memorequip hint example

Next, I added a funkier option for getting an at a glance idea of where you stand with all of your "memories." Previously, the "Your Memories" page (also previously the home page) would give you a table with a bunch of numbers on each memory. Number of attempts, worst/best/last scores, and a predicted next score were all listed. Personally though, I find it hard to read at a glance. So I came up with something a bit more abstract but in my opinion prettier.

You could maybe think of the new visualization like drawing on a measuring stick with markers that fade over time. The right end of the stick represents perfect memorization while the left end represents complete ignorance of the text. Each attempt at the text will add a new mark on the stick. Older attempts will fade as new attempts come in. The color of each mark shows whether it was an improvement (green) or a regression (red). And lastly, the width of each mark shows how much of an improvement or regression it was.


memorequip attempt history table

I like the idea of my past performance fading away because it means I can in theory hide those bright red regressions behind more recent bright green improvements. It encourages me to try to retain my memories but also is a little forgiving when I don't. And since the brightest mark is for the hypothetical predicted next score, I can get a pretty good idea what texts could use some attention.

All that said, the "classic" tabular view is also still an option you can choose in settings if that's more your style.

Finally, I took some steps toward making the site more mobile friendly. Along with trying to make a responsive design and UI, I also fixed a few bugs that actually affected the functionality. Main thing here is just that the sidebar navigation becomes a top bar dropdown menu at a small enough screen size. It could still be better, but it seems to work for most of my purposes.


Rated T for Teen

I may have also added a new, potentially controversial game thing to the site. I call it Reference Hangman. And I actually built it several years ago but felt like it might be a little much for the Scripture memorization crowd so I kept it to myself. But basically the idea is the reverse of the normal memorization mode. Instead of being given a Bible reference and having to type out all of the text, you are given all of the text and have to guess the reference. I definitely feel the need for this mode because with something like 150 texts I wish to simultaneously have memorized, mentally linking a text to a reference becomes quite important. In the normal mode, I rarely find the reference all that helpful and often need to take a few hints to get me going.


hangman game teaser

As you miss your first few guesses, a little animated dude appears on your screen along with a noose and such. If you continue to make incorrect guesses, the noose lowers and picks up the poor guy. At which point, you have 30 seconds to figure out the reference before he stops breathing. On the bright side, as you make incorrect guesses, the options you have to choose from are filtered down.

And yes, as the heading suggests, I did actually make it so 12 year olds and younger can't get to this game because why not. That said, I probably played more hangman under the age of 12 than over so feel free to hit me up if you're outraged.


Home Page Charts

Lastly, I started a new home page that I hope to turn into a little dashboard that will provide a better overall look at how you're doing with your memories. In addition, I also have included charts that show the performance of the model that predicts your next score.

On the user side of things, you can see how well memorized all your memories have been in the past, the present, and the future (in a worst case scenario where you don't attempt any of your memories for 6 months). This shows a better overall trend than you can get from the "Your Memories" page. Also since the predictions for the future are aggregated over all of your memories, the inaccuracies in the predictive model should largely offset each other - predictions that overestimate your memory should offset with predictions that underestimate your memory. Also, flattening that predicted forgetting curve is a nice goal to have. If you consistently spent something crazy like 30 minutes memorizing every day for a year, you'd likely see your predicted retention be much higher than if you put in 10 minutes every month. And the dropoff of that predicted forgetting curve ought to be a solid representation of your retention.


User Stats


percent memorized chart

attempts chart

active memories chart


Predictive Model Stats


model inaccuracy over time chart

model inaccuracy by attempt number chart

model inaccuracy by time since last attempt chart

On the model side of things, mostly the charts are there for me to monitor its accuracy so I can determine if anything unusual is happening. But it also serves as some nice transparency into the imperfection of the model. It will make large mistakes sometimes. But don't worry, I've programmed the AI to never harm a human. And even if it does, it will at least learn from its mistakes. Which, really, what more can we ask for? Sorry, obligatory jokes about how I probably accidentally built Skynet. Anyway, the chart with the moving average of the accuracy of the model should ideally decrease as more data becomes available for it to generalize with. Which is why, as you can see, I didn't attempt to memorize anything for all of 2016 and half of 2017 - because I wanted it to have a diverse dataset to learn from. At least that's what I tell myself.

Finally, here's a shorter, more data science-y article about a new hint I added to MemorEquip: TL;DR: TFIDF is GR8 IMHO