Big Change for Users

We started Kountr, simple. When first asked for persistent features we opted to use sessions and client side cookies to achieve these features. The Your Kounts and Your Recent sections were achieved this way.

Recently we added the notion of users. When you joined we would sweep up all the kounts you “owned” and make this right in the database. For the last couple weeks I have been trying to maintain these two seperate systems for ownership of kounts.

Bugs and new feature requests have pushed me to remove the cookie based system. Now if you want to own your kounts, then you need to join the site. It is just about the easiest site to join. Provide a username and email and you are immediate joined and logged in. We email your password for the next time you need to login.

If you join Kountr and would like to take ownership of your kounts, just send us some feedback, which kounts, and we will make it right.

We appreciate your patience during these bumpy changes and hope you will enjoy and be inspired by the new features we are developing.

I Just Knew

I have had a data model for all of a week, maybe. And I want to replace it all. It just doesn’t scale. I have three ways to deal with the three different security related features I have implemented. So tonight I will do the biggest DB mod and data migration I have done to date. This will be a tough one. I’ll try to keep some notes going here as I go along.

Step one, take down the site.

Most important technical thing I learned this week;

INSERT INTO table_name ( col1, col2, … ) SELECT ( col1, col2, … ) FROM table_name2;

This is just what I needed for my data migration. I am normalizing the DB so I needed to move partial data from one table to another. This saved my life. I might actually get to bed before 2 am now.

Second best technical thing I learned this week;

UPDATE table_name SET col = other_table.col2 FROM other_table WHERE …

So awesome! Inline Datamigration is a total possibility.

Third;

SELECT * FROM table1_name t1, table2_name t2 where t1.col1 = t2.col2;

I am such a SQL newb.

Fourth, a ringer really, I learned this last week but didn’t blog about it.

Django model managers are awesome. They enable you to create much cleaner code. Basically you can move any DB related processing into the manager so that your views are written at a higher level. I will write about this more in the future.

I went live, had a couple issues, but overall that was a pretty smooth data migration. Quick lessons learned once the site was live;

1. manager functions are not accessible by instances
2. don’t forget to include new models in your view files
3. the filter function cannot take expressions, not sure I am properly understanding how this works

This was a pretty good evening. Started the migration around 10:15pm, it is 1:15am now and I am heading to bed. I have tested the site for 10 minutes but I need some real QA after I do something as big as this.

My next step will be to move iKount, and Privacy over to this new model. Then I will implement the visibility functionality with the same model. Some other cool features are just on the horizon.

Django’s “order_by”

Django is awesome for all of the “we’ve already done it so you don’t have to” features. So I am always caught off guard when it falls short.

When you have a QuerySet you can use order_by to order the set before display. This is really handy. Unfortunately, you cannot instruct it to ignore the case of the strings. Debugging this problem was even harder because I stored the strings in the DB as they were entered by users, but only display them in all lowercase on the website. I couldn’t figure out why the list was in two parts, both sorted correctly.

As a fix, I have set the strings to lowercase before they are written to the DB, and gone and changed the current ones to all lowercase. The SQL looks something like;

update table set column=lower(column);

Overall not a huge problem, but it would be pretty slick if Django allowed you to do;

Count.objects.filter(…).order_by(‘name’, ‘ignorecase’

Hope I saved you some anguish.

Unhandled Exception

Never again.

If the browser gets an “Unhandled Exception” that means there is an exception occuring on the server that is not being caught. I guess many get caught by django and display as the normal error page, but there are some, I don’t know which, that sneak by and leave a totally useless error page.

In this case I had the added frustration that it didn’t happen on every computer, and it didn’t happen on my development box. I saw the problem on someone else’s computer, sent myself a note about it. When I got home and checked the problem was “solved” so I chocked it up to server error. It wasn’t until the traffic to the site fell of the face of the earth, and then a user sent me an email to say “what gives”, that I figured out there was a problem.

The problem is fixed now.

Security Model

Now that I have users, I can have the notion of ownership, and once granted ownership users will want to play with visibility and share with limited groups. This all gets very excited.

So what is the correct data model?

This post is just a mental note to self, of my current reasoning. I’d like to sleep on it before implementing.

Keep It Simple

Flat table, user_id, kount_id, permission_type, all ints.

It is a simple lookup to see if someone has the right permission. If the types of permissions permutates then I can do that without DB changes.

The open question is; Does this model scale? and how far?

I am a long way from needing to scale, and a simple, flat table should be relatively easy to migrate should a better, more scalable model, present itself.

I will ask one smart person about this before implementing. (This is a promise to all my future users who will not suffer from the capacity type problems plaguing Twitter)

Stay tuned, I’ll write more about this topic, soon.

A Confession

I have not had a proper backup routine in place for Kountr.

If the hard disk on my shared, remote server were to have been lost, all of the content on Kountr would have been lost with it. I have the application stored locally, but the DB is in one place.

I have setup a cron job on the server to dump the db nightly. I will run another cronjob on my computer to copy these dumps daily. Once the first one is complete I will go through the restore routine to make sure everything works.

In case anyone else is bootstrapping their own web app and don’t want to take the time to figure out the nightly back up here are some tips.

1. To create a nightly cronjob;

export EDITOR=vi

crontab -e

20 3 * * 0-6 command

This will run command at 3:20am everyday

2. Command for dumping a db and zipping it;

pg_dump -F t DBNAME | gzip > FILENAME

3. Getting a date in the filename;

Insert $(date +%d%b%Y) in your filename string

Good luck.

I Guess that Makes it Social

So I have finally exposed the user functionality. You can join, login, and logout. That is the extent of it. I have a back log of requests and bugs related to the user login functionality. I’ll be working on this over the next couple days and we will be doing a huge marketing push once all that is in place.

Try it out and let us know what features you want to see now that you have an identity.