.ofocus file oddity - why so slow to sync?
 

Okay, as far as I can tell the .ofocus file is just a package for some xml, and it's tiny, even with a fairly ginormous number of actions.

Yet when I sync my .ofocus back and forth between iMac<->MacBook Pro, it always takes quite a while to copy the file, a good minute or two, much longer than for an MP3 or a huge picture file ... which are much larger files.

I sync using ChronoSync, and other files take the expected length of time to copy, so I don't theeenk it's a ChronoSync issue.

Anyone know what's going on here? It's not really a problem, I'm just bepuzzled.

I am wondering about the format of the .ofocus file, too.

I discovered when I upgraded to 1.0.2 (and thought my .ofocus file was corrupt) that my back ups weren't really backing up. I use FileSync <http://www.designersdomain.com/software_files/FileSync.html> to back up my files to a remote server. When FileSync does its thing, it lists all the files it is copying over to the server. When it gets to the .ofocus file, it lists HUNDREDS of individual files. If you open the package contents of the .ofocus file, you can see all these individual files (and if you are brave and try to open them, at least the way I did it, you get a bunch of computer symbolese nonsense). When I went to use my supposed back up, the .ofocus package was empty! (Which led to a morning of despair thinking I had lost all my OF data, but things turned out fine... see this thread <http://forums.omnigroup.com/showthread.php?p=36924#post36924> for details).

So I am wondering why all the individual "files" within the .ofocus package (not sure if this is the right terminology) wouldn't copy over to my back up server? Any thoughts or ideas?

Thanks,
Eric

Every time data changes in the OmniFocus database, it gets written out to disk as an individual transaction file, stored in a folder that the finder knows to show to the user as if it were a single file. (Special folders like this are called "bundles"; it's a feature of Mac OS X.)

Practical example: adding an item to your OmniFocus database, changing it, and then deleting it all get written out to disk as three files, and those files stick around. When you delete the item, the previous two files are still on the disk.

On your hard drive, this makes a lot of sense - it's very speedy to write those small change files, if your computer goes kaboom we only lose the transaction file we're writing instead of the whole database, and other things that require more smarts than I possess to explain. Ken explained them elsewhere on the forums... someplace.

When it comes time to send your database to another machine, though, the file transfer spends time sending one of the transaction files, closing out that file, opening the next one, sending it, then closing it; repeat over and over again. (MP3 files are all one big stream of data - open the file, send all the bits, close the file.)

If you select File -> Rebuild Database, we'll collapse all those transactions down into one big 'State-of-the-Database' file, which will be faster to send over to the other machine.

[QUOTE=webalstrom;37004]So I am wondering why all the individual "files" within the .ofocus package (not sure if this is the right terminology) wouldn't copy over to my back up server? Any thoughts or ideas?
[/QUOTE]

Yep, that's the right terminology - bundle or package are somewhat interchangable. I can't explain why the files didn't back up, though. That would seem to be a problem with the file sync application.

Or are you saying that the bundle was empty on the machine you were transferring the file away from, before you synched?

If that's the case, the bundle shouldn't have been [B]completely[/B] empty; there should be a single XML file inside. To clarify the previous post - when you rebuild the database, we collapse all of those transaction files down into one XML file. Unless you modified the backup, there would be one XML file and no transaction files.

[QUOTE]If you select File -> Rebuild Database, we'll collapse all those transactions down into one big 'State-of-the-Database' file, which will be faster to send over to the other machine.[/QUOTE]

Would it be okay to rebuild the database every week, when I do my weekly back up? Or is rebuilding something which should only be done infrequently for very specific database maintenance reasons?

[QUOTE]I can't explain why the files didn't back up, though. That would seem to be a problem with the file sync application.

Or are you saying that the bundle was empty on the machine you were transferring the file away from, before you synched?

If that's the case, the bundle shouldn't have been completely empty; there should be a single XML file inside.[/QUOTE]

No, the bundle on my computer wasn't empty, at least I don't think it could have been since its my active .ofocus bundle I use everyday. But on the backup server, the file is 0KB (zero) in size and when I show package contents it is totally empty. I know when FileSync is running I see a LONG list of files being copied over (and I know they are .ofocus bundle files because they have the same names). But nothing is actually copied to the bundle. Another backup of the .ofocus file I zipped first was fine upon copying back to my computer and unzipping.

The only thing I can think of is the server I am backing up to is a Windows (or at least a non-Mac) server. I have to use a "SMB://" file path name to access it. Could this be a problem?

Whatever may be going on, I am going to make sure I zip up my .ofocus bundle before I back up as well as make a ".taskpaper" back up to in order to be doubly sure!

Eric

[QUOTE=Brian;37017]If you select File -> Rebuild Database, we'll collapse all those transactions down into one big 'State-of-the-Database' file, which will be faster to send over to the other machine.[/QUOTE]

Wow, cool: I heart Omni. I post a question that feels a bit obscure and trivial, and damned if I don’t actually get an articulate and complete answer!

And, yes, as you predict, syncing my .ofocus [I]is[/I] much faster after rebuilding — practically instantaneous, actually. It’s not that it was arduously slow before, but it [I]bugged[/I] me that I didn’t understand why it was sluggish. Thanks for scratching that itch, Brian!

[QUOTE=webalstrom;37042]The only thing I can think of is the server I am backing up to is a Windows (or at least a non-Mac) server. I have to use a "SMB://" file path name to access it. Could this be a problem?

Whatever may be going on, I am going to make sure I zip up my .ofocus bundle before I back up as well as make a ".taskpaper" back up to in order to be doubly sure!

Eric[/QUOTE]

This sort of thing isn't uncommon: Windows servers tend to have problems with bundle files sent over from Mac OS X.

Yeah, zipping the file up before synching is probably the way to go here.