[scaleslea]

My wife and I have been having lots of problems with long sync times both on the iPhone and the desktop client when syncing to iDisk. It wasn't until today when I finally ran the Coalesce database script that we finally got sync times measured in seconds instead of minutes. All this time we've been wondering what was wrong with OmniFocus and why it was syncing so slowly.

So I've been wondering, what kind of average sync times is everyone else getting?

Today my sync time went down to 20 seconds from 2-5 minutes.

Doc

[ohammersmith]

I'm finding that I have to run the coalase database script anytime I do a lot of editing on my desktop OmniFocus. I've been tracking the size of the folder on the server, along with the types of files in it. Yesterday I was catching up on my weekly review (3 weeks behind. bleah.) and my file exploded up to 3 megabytes. No amount of syncing could it it to come down very much. This made my iPhone sync times a copule minutes, or more.

These are the graphs I have from my monitoring system.





I think the solution is to sync less often. Very rarely is the every minute sync going to be useful in my use cases. I still have to be sure to sync before I close my laptop, or lock my iPhone. All every minute does is create a crazy number of transactions to catch up on... you can see from the graphs that I got up to 600 transaction files.

[whpalmer4]

If you don't have some stale client entries (check the Show Clients display in the Sync preferences in OmniFocus), two syncs on all of your devices separated by an hour should cause your database to automatically compact.

[ohammersmith]

Originally Posted by whpalmer4

two syncs on all of your devices separated by an hour should cause your database to automatically compact.

Why the hour? I guess there's something OF is waiting for that I don't realize?

I'm pretty sure I did that yesterday, but I can't guarantee it. A number of the transactions were at least 2 hours old, I do know that.

[whpalmer4]

That's the way the code makes sure there isn't a really slow client sync in progress, or some other condition that would make it unsafe to compact the database. If everything is synced up twice, separated by at least an hour, the compaction code should run.

[ohammersmith]

Cool. Next time it blows up to 3 meg, I'll be more patient and see if the database compcats itself.

[uof]

Regarding the hour gap between syncs: does anyone know if you sync on any client before the hour is up (just a sync, with no changes made to the file) stops it coalescing until you get a full hour gap?

[Lizard]

It shouldn't keep coalescing from happening, you just won't get quite as much coalesced.

For example:
Day 1
9:00 sync all 3 clients -- 15 .zip files on server
10:05 sync all 3 clients -- 18 .zip files on server, deleting all client files before 9:00
10:10 sync 1 Mac -- coalesce brings you down to 4 .zip files

Day 2
9:00 sync all 3 clients -- 15 .zip files on server
9:30 sync iPhone -- 16 .zip files on server
10:05 sync all 3 clients -- 19 .zip files on server, deleting all client files before 9:00
10:10 sync 1 Mac -- coalesce brings you down to 5 .zip files

[eecue]

Can you post the scripts for that cacti graph? Thanks!

[ohammersmith]

Originally Posted by eecue

Can you post the scripts for that cacti graph? Thanks!

It's actually a munin plugin, but yes, here they are. They are very simple and not set up for being any old user. You'll have to edit in your username in the script.

If I was feeling ambitious (or needed it), I'd rewrite it so you could symlink to omnifocus_disk_ to omnifocus_disk_<username> but I don't really have a need for that, right now.

FWIW, I've seen that OmniFocus really does compact things correctly, I just wasn't being patient enough... or syncing my iPhone often enough. I find it makes things better if I happen to open my iPhone and sync in the middle of a major multi-hour edit session (review day, etc).

Anyway, scripts.

Disk space usage graph: http://pastie.org/303332

Number of files and types stacked graph: http://pastie.org/303335

You'll note from the comments that the latter was modified from the apache process graph. :)

BTW, Pastie seems to to be repeating the here doc boundaries. Make sure you download the text version or you won't have a working script.