Just Journal Developer Blog

The hard drive died in the compaq server for JustJournal and my personal site last tuesday. The /usr partition was toasted including the mysql data files and all of the website data. fsck couldn't do anything with it.

In summary, the mysql data was destroyed containing journal entries. The most recent backup that was salvagable was from January 3rd, 2004. Anyone that signed up after that date or any journal entries posted after that date have been lost.

The source code for justjournal is backed up on my ibook and i was able to publish it back to the new hard drive.

how did it happen?
The 40 gig seagate IDE drive (OEM) failed. It appeared there were bad sectors on the disk. Modern IDE drives have extra sectors not usable that are automatically replaced when the drive detects problems. The limit was reached. Justjournal is housed at work and currently runs off of a DSL line on my PC running FreeBSD. The furnace has been out most of the winter and the roof has been leaking in the server room. These conditions are VERY bad for hard drives. There isn't much i can do about it because quite frankly I can't afford to get another colo setup like I had pre-november. The server also provides mail services for work and they were gracious enough to buy me a new drive since it was probably their fault.

I lost my data.. what are you going to do about it?
Not much. I can't go back in time and backup data more often. My mysql server has a root password and therefore i can't automate backups. I refuse to put a root password in a shell script. Its just dump. I may try to setup a user just for backups with read access. In the mean time, i will start backing up mysql and the jj webspace more frequently. I lost the web.xml mapping file as well which i had to rebuild. It is entirely my fault for not providing regular backups. In addition, I plan to start using the CD burner once a month to backup data in tar format and possibly using the unix dump command.

I also plan to add a way users can backup their own journal entries and retrieve all of their data so its not entirely on my head when this happens. I can't afford a tape drive or raid arrays as I don't make anything off of my websites including justjournal and foolishgames.com.

I appoligize for my mistake and hope to improve my administration skills in the future.

Also, there has been some loss of functionality in the short term. Posting and most of the preference systems should be back up tonight. Several tables are missing that weren't yet in existance in january. Comments are not working right now. I will try to get those working very soon.

I fixed the calendar problem. One of my classes assumed that java would count months starting at 1. Since its january, that was bad!

While I was in the code, I optimized it and added mood icons and security icons like the recent entries and friends pages.

I also added code to let you look at previous years entries.

That code is behind with friends security. It should only display public entries to someone on your friends list. I'll fix this soon.

I also did some research and found several bugs in mysql and tomcat 5. Those issues could cause unexpected behavior in jj. I'm trying to make the code more robust to report errors and deal with them gracefully. I've had to add checks for conditions that should never happen which slows down the site, but at least it doesn't crash the site!

I also tuned the java runtime parameters to speed things up. Java can take advantage of optimizations in the server vm and a few other memory settings.

For some reason, the calendar feature is broken. Its possible with the new year something was broken.

I'm looking into the problem.

It is now possible to edit and delete comments if you are the owner of the comment. Spell checking comment edits is coming soon.

I also added some indexes to speed up sql queries.

JJ has a new view comments page. I also added the ability to delete comments (for the poster).

Spell checking comments is about 2/3 of the way completed as well.

editing comments will be added soon. The database access code is already written.

editing and deleting journal entries is on a high priority list.

I've been working on the profile pages tonight. I have serveral bugs worked out. I removed AIM indicators which no longer work and added ICQ indicators.

I want to add these features for all services but that will take time.

I've been working on the backend of the comments system over the weekend. Hopefully i'll get that fixed up and will have new profile and comment features ready to go this week.

Caryn is working on a protocol for a possible JJ client.

The spell check bug is not in my code per se, but rather in the spell check engine itself. It returns altered strings. I'll try upgrading it and digging in the source this week.

I just uncovered a bug in the spell check engine. If an apostrophe is used, the entry is not added properly. This also caused problems accesing the database for the user after that time.

In addtion, the spell check entry doesn't get cleared out if there is an error.

I'll try to resolve both of those problems soon.

Henry is now the official spokes-cat for Just Journal. He has been living with me for two years now. He has quite an appetite at 17 pounds!

Henry often sits on my lap when i'm coding the site. It was only fitting that he would promote it. :)

Users can now change their password. Go to preferences while you are logged in to change your password. Remember passwords must be larger than 5 characters.

A-Z
a-z
0-9

(special characters may work, but i won't guarantee i'll support it forever)

I fixed the comment system. Entries now consistantly appear. I'm planning on adding some management capabilities for journal owners soon.

Just Journal now uses 1 way hash encryption to protect passwords. Basically your password is not stored anymore, but rather a hash of it. (a 40 bit hex number representation) I can't give out passwords anymore but i can still reset them.

The reason for the change is so that if someone hacks into the mysql database, they can't get your password. Some people use the same password at multiple sites. This is a new trend in web site security. Logins were down for the last two hours for some users as the passwords were hashed and the code was uploaded.

I still need to fix create accout to use the new hash system.

I also wrote code to change passwords. I'll get that live this weekend.

Friends relationships are working on the profile pages. You can now see your friends and people who added your journal.

Note the profiles are still rather buggy. I found a bug with the SQL driver tonight and had to alter the database logic for the profile to get them to work more consistantly.

Profiles are still cached by the web browser from time to time. I'll need to make a servlet or something to fix that issue. Also, several of the fields are not alterable yet because they have not been added to preferences. (AIM, ICQ, etc)

It is now possible to spell check a post. There are a few minor bugs with the system yet. The location and date are not saved during the spell check transaction, but the other fields are saved.

The dictionary is US English only. I need to get dictionary files for other languages including British English, German, French, Spanish, and any other languages. I'd be happy to take contributions.

If the spell check engine doesn't recoginize a word, it will not report a misspelling. Feel free to submit words to add to the dictionary file.

In the last 24 hours, i've added new mood icons, spell check, and a few bug fixes. I also increased the size of the entry input form.

On a friends page, if an entry has security attached then the proper icon is now displayed.

That way your friends will know the entry is private and not to talk about it publicly! (well hopefully)

Remember that comments are technically public. Unless someone can see the entry, they can't click on the comments link though. It is still possible to view them if they guess the comment id or someone gives it to them. (i may change that in the future)

Friends security is partially working. You can now view your friends secure entries when logged in on your friends page.

Only bug is that you can not view our own entries on your friends friends page yet. (that is not a good sentence, but its 2 am)

The SQL query is a bit odd that i wrote. If anyone has entries go public that shouldn't or find a bug in the system let me know.

I've accomplished a great deal tonight with the mood icons. The icons are now displaying for seven root categories and are inherited by sub categories. There are 28 total root categories which means 21 need icons for all moods to have icons.

I have the icons working for individual journals and friends pages. Previously, no icons appeared on friends pages.

In addition, I fixed the bug where "Not Specified" appeared for location and mood. Those fields now only appear if the user selected a valid item.

The new mysql version is slower and the additional overhead of mood icons has slowed down the site. I'll work on optimizations over the next week. I started consolidating code in september, but the college semester through me off track.

During my break I hope to fix all the outstanding bugs and broken features that the public knows about. I want to get spell check and friends security working.

I also need to work on another site I actually get paid for so i might not get everything done during break.

The spell check engine has worked since august but i never created the jsp pages to use it.

Earlier today, the database and webserver were taken down for upgrades. The database upgrade hit a few snags. Mysql protocol was changed in the 4.1.1 release which caused the site not to work for some time.

I have installed a new jdbc driver and fixed some compatibility issues.

I'm installing a new version of FreeBSD today. The server should only be down a few minutes. I know this site is in great need of new features and hopefully i'll get to it during the holidays. My college semester ends next week, and I am the only developer.

JustJournal was moved to another server over the last few weeks. The new setup is a bit slower, but much cheaper.

When the speed becomes an issue, I'll look into something faster.

I've also been looking for bugs in the existing system. I have a small list of things that need to be corrected (plus spell check and mood icons).

The server has been upgraded to a new release of freebsd and native java 1.4 p4. The java upgrade took longer than expected.