Wednesday, October 14, 2009
J's and U's Updated / Speed Increases
I mentioned a few weeks ago that I planned on making I's/J's and U's/V's look the same on the back-end, while preserving their traditional orthographies on the front-end. I've just completed this task!
My main motivation for making this update is because certain passages stored in The Latin Library reflect the older conventions of using J's for consonantal I's or U's for both consonantal and vocalic V's. Numen's parsing engine was having trouble recognizing forms like jecit (iecit) and uuius (vivus). So now as a result -- after a bit of work -- the engine is updated and now recognizes more possibilities than ever. Incidentally, internally J's are stored as I's and U's are stored as V's.
Another project I completed at the same time is an order-of-magnitude speed improvement for parsing. I was trying to figure out ways to make the engine faster and I discovered a shortcut that boosts speed tremendously. When parsing a word, the engine used to spend between 250ms and 500ms parsing each word! That was always disappointing to me, but I had gotten around the problem by caching the results. Now, however, word parsing takes about 25ms!
Why bother improving the speed? Because soon I will be implementing word lists and frequency lists! A word list, of course, is just a "mini-lexicon" that defines only the words in your chosen passage, and a frequency list is a list of words in order of how often they appear in a passage. The word list will be helpful to quickly work on vocabulary for a passage, and a frequency list will help Latin students study more effectively by giving them the most frequent words first. I'm very excited about this feature, but I don't anticipate it will be done before January 10th (giving me the winter holiday to work on it).
That's all for now!
My main motivation for making this update is because certain passages stored in The Latin Library reflect the older conventions of using J's for consonantal I's or U's for both consonantal and vocalic V's. Numen's parsing engine was having trouble recognizing forms like jecit (iecit) and uuius (vivus). So now as a result -- after a bit of work -- the engine is updated and now recognizes more possibilities than ever. Incidentally, internally J's are stored as I's and U's are stored as V's.
Another project I completed at the same time is an order-of-magnitude speed improvement for parsing. I was trying to figure out ways to make the engine faster and I discovered a shortcut that boosts speed tremendously. When parsing a word, the engine used to spend between 250ms and 500ms parsing each word! That was always disappointing to me, but I had gotten around the problem by caching the results. Now, however, word parsing takes about 25ms!
Why bother improving the speed? Because soon I will be implementing word lists and frequency lists! A word list, of course, is just a "mini-lexicon" that defines only the words in your chosen passage, and a frequency list is a list of words in order of how often they appear in a passage. The word list will be helpful to quickly work on vocabulary for a passage, and a frequency list will help Latin students study more effectively by giving them the most frequent words first. I'm very excited about this feature, but I don't anticipate it will be done before January 10th (giving me the winter holiday to work on it).
That's all for now!
Labels: accuracy, database, development, features, frequency lists, google cache, orthography, parsing engine, performance, slowness, vergil, word lists
Saturday, August 29, 2009
New Server
I apologize for taking down Numen for a few hours! But it was for a good reason. A couple of weeks ago (as I posted) the server died, and I had to jury-rig a (slow) home computer to be the server.
Well, just yesterday a bunch of new parts arrived via UPS and I quickly built, tested and verified the new server. I took down the old one about 11am, copied over all the data, and fired up this new one just a few minutes ago.
The computer power users might be wondering what the new server consists of. It's a very simple configuration. It's a dual core Athlon 245 (?) Athlon II 2.9GHz processor, an nVidia-based AM3 motherboard, 4GB of DDR-2 1066, an Antec EarthWatts 380W power supply and a WD Green 1TB hard drive. I was specifically trying to build a power-sipper here. My best guess -- because I forgot to bring a Kill-A-Watt to test it, is that this server runs somewhere around 85W and around 110W at peak. I wanted to get an Intel SSD (160GB Generation 2), but ... I felt that the performance gain from that wouldn't outweigh its cost (around $420). Now why does that matter? Because this entire computer, including shipping and a $20 mail-in-rebate was about $292! Nice!
This new box is quite a bit faster than the home computer which was the temporary surrogate, but you'll be happy to hear that it's also zoom-zoom-zoom fast compared to the original server! It's somewhere in the range of 2 to 3 times faster overall.
The computer power users might be wondering what the new server consists of. It's a very simple configuration. It's a dual core Athlon 245 (?) Athlon II 2.9GHz processor, an nVidia-based AM3 motherboard, 4GB of DDR-2 1066, an Antec EarthWatts 380W power supply and a WD Green 1TB hard drive. I was specifically trying to build a power-sipper here. My best guess -- because I forgot to bring a Kill-A-Watt to test it, is that this server runs somewhere around 85W and around 110W at peak. I wanted to get an Intel SSD (160GB Generation 2), but ... I felt that the performance gain from that wouldn't outweigh its cost (around $420). Now why does that matter? Because this entire computer, including shipping and a $20 mail-in-rebate was about $292! Nice!
Okay, enough geeking-out for now. I hope you enjoy using Numen, and as always, feedback is welcome!
Labels: performance, web server