8:15 PM - It was newsletter day again
One of the tasks I've been given at work is to write and maintain a newsletter generation tool. Most people think of it as a simple application that sends and email with some canned text, maybe customized with the persons name similar to the "mergemail" feature in word.
Well that's only part of what this application does. It generates four different formats of newsletter, HTML, Text, PDF and a special HTML version to send with the PDF. The PDFs are 50 pages long for one client everyday. Today, I had to add stock price monitoring to the newsletter tool. It checks the price of the stock on regular intervals (hourly) and includes the current price when the newsletter is sent out for this client. I spent way too much time on this, but I have a feeling it will come up again.
Like everyone else on the Internet, I used Yahoo! Finance data as a source. You can output a CSV directly from their web app and even query multiple symbols. I limited it to hourly lookups and then only during market hours using a cron job. This is probably the 7th cron job running. They're all Java apps right now.
I'm really torn on the whole Java thing with this. I think Perl or C would be better choices. However, since the database schema changes constantly, I just used a jar file containing all the database stuff and pop it in when I do a schema change. It really simplifies updating them.
I hate spec changes, but it's common place at my company. You can never get someone to tell you the whole story at any time or even half the story. When you do get the story, you find it's changed from the week before.
Anyway, the stock fetcher is pretty simple and slick. I just used a Java Url class to fetch the data and then an open source library (LGPL) to parse the csv. It's easy to parse CSV files yourself, but I didn't feel like taking the time.
It runs through a loop and checks for each symbol stored in the database with a 30 second pause so as to not klll Yahoo's service.
I had a fun bug today using html clean and jtidy together. It seems jtidy was re-encoding the HTML entities again like ampersand so it was screwing up non breaking space, greater than, less than and apostrophe characters in the HTML output. I have to run both because jtidy sucks. It doesn't fix several types of invalid HTML. If I ever get time, I'll try to fix it and upstream it.