The problem described in this [URL="http://forums.omnigroup.com/showthread.php?p=69207#post69207"]thread[/URL] sounds similar. Looks like reseting your iPhone database gets sync times back to normal. It doesn't help much in getting to the bottom of this issue, but at least it should get you back on your feet again. Just be sure to have all your data synced and a good backup before beginning this procedure.

-Dennis

Dennis says:

[QUOTE=Toadling;69065]my OmniFocus database rarely takes more than 5 seconds to sync over MobileMe on a Wi-Fi connection. I have two clients (a Mac and an iPhone), both are synced a couple times a day, and my database has around 2200 actions and 330 projects with a file size of about 500 KB (no attachments).
[/QUOTE]

I would really like to know how you pull this off. I don't think I have nearly that many actions or projects, and today I'm already 10 minutes into a sync and wondering how soon it's going to finish so I can get out of the house and start my day. This is nerve-wracking.

Can someone please post that OG link to the thread or sticky that describes how to pare down the zip files so that syncing is sped up? I can't find the procedure in my notes. Thanks!

[QUOTE=santra;69701]I would really like to know how you pull this off. I don't think I have nearly that many actions or projects, and today I'm already 10 minutes into a sync and wondering how soon it's going to finish so I can get out of the house and start my day. This is nerve-wracking.[/QUOTE]

I haven't done anything special other than avoiding attachments and syncing both of my clients regularly. It's been largely maintenance free.

In fact, I've [I]never[/I] had particularly long sync times, and I'm wondering what it is that's giving some people so much trouble in this area.

As for compacting your database, you could just sync all clients and then unregister them. On one of your Mac clients, go to Edit -> Compact Database. When it's done, turn syncing back on for that Mac client. When it's done, turn syncing back on for your other clients and be sure to choose "remote copy" or "server copy" of the database (not sure how that dialog is worded now).

-Dennis

With all due respect, people here don't want talk to "Support Ninjas." They want a new release that fixes the problem. This issue has existed for the entire life of the iPhone Omnifocus, and it makes it a lot less useful. If you are like most people and have a few hundred actions, and use the desktop much more often than the iPhone, sync on the iPhone will take on the order of minutes. The most common use case for the iPhone app is to either quickly add an item to the inbox for later processing, or look at what's available in a particular context/project, and then close the app. What should take a matter of seconds takes minutes, because the user has to wait for Omnifocus to sync down, and then if an action was entered, sync back up to the server. The amount of data that changes from one sync to the next is always just a few kilobytes, and so connection speed and the iPhone's processor should be more than adequate to do this job in a matter of a couple of seconds at the absolute maximum.

[QUOTE=kammeyer;70802]If you are like most people and have a few hundred actions, and use the desktop much more often than the iPhone, sync on the iPhone will take on the order of minutes.[/QUOTE]

I've got a few [I]thousand[/I] actions and use the desktop much more often than the iPhone. My sync rarely takes more than 5-10 seconds on a Wi-Fi connection.

There may well be a common problem here, but I'm not convinced it's as widespread as one might think. I've had virtually no problems with OmniFocus syncing since day one.

-Dennis

[QUOTE=kammeyer;70802]With all due respect, people here don't want talk to "Support Ninjas." They want a new release that fixes the problem.[/QUOTE]

I'm really sorry for the frustration this has caused, and what you're looking for here is totally reasonable. Unfortunately, there isn't a single problem, so we can't make one release that fixes everyone's troubles.

So, what do we do? Every time we can narrow down a problem that makes sync slow, we fix any problem in the code and ship it as soon as we can. We also try to post here on the forums to explain the best ways for everyone to make their syncs as fast as possible, if it's not something that a code fix will address.

It sounds like your syncs have been taking a long time, and it's been that way for quite a while. Our intent is absolutely not to have that be the case. The tips [URL="http://forums.omnigroup.com/showthread.php?t=12071"]here[/URL] have helped a lot of customers; when they don't help, we're pretty successful at getting people back up and running via phone or email.

In the cases where the ninjas are truly stumped, it means that we're working on a problem we haven't seen before. Gathering information about the problem means the dev team can work on fixing it so everyone can benefit from that fix in our next release.

I can't emphasize enough how much we'd like to get this working well for you. I wish you didn't have to take the time to contact the [EMAIL="omnifocus-iphone@omnigroup.com"]support ninjas[/EMAIL] to make that happen, but it really is the quickest and best way to get this taken care of. I really hope this helps.

[QUOTE=Brian;70826]
I can't emphasize enough how much we'd like to get this working well for you. I wish you didn't have to take the time to contact the [EMAIL="omnifocus-iphone@omnigroup.com"]support ninjas[/EMAIL] to make that happen, but it really is the quickest and best way to get this taken care of. I really hope this helps.[/QUOTE]

First, let me say that I love Omnifocus. No other app compares in power, and using it has significantly improved my life. It is worth at least 50x what I paid for it. The availability of Omnifocus was a signifcant reason I switched to the Mac platform.

That said, I have followed the advice here, and it always does help temporarily, but the problem always returns. The problem is that the sync problems are not fundamentally a support issue, they are a software architecture issue. Telling users to contact support as if it is some problem with their configuration rather than a problem with the software itself is disingenuous.

As for the architecture itself, I'm not sure where the biggest problems come from, but some aspects are obvious from looking at what drops on the webdav server.

1. The data files are too large and too complicated relative to what they are actually holding. Just adding an item to the inbox requires creating one or two XML files on the server, which contain tons of extraneous information. Just the file that contains the item contains all sorts of information about my computer. Here's the file generated from me adding a single two word action to the inbox:

<?xml version="1.0" encoding="utf-8" standalone="no"?>
<omnifocus xmlns="http://www.omnigroup.com/namespace/OmniFocus/v1" app-id="com.omnigroup.OmniFocus" app-version="77.41.6.0.121031" os-name="NSMACHOperatingSystem" os-version="10.6.2" machine-model="MacBookPro4,1"><task id="apXSog3buY1"><inbox/><added>2009-12-17T07:37:31.077Z</added><name>Test Action</name><rank>2080374784</rank><order>parallel</order></task></omnifocus>

2. The app creates a new XML file every time something changes, and then because the XML files are so verbose, it ZIPs them up. Thus, for every new action, the phone has to make an HTTP connection, download a ZIP file, unzip it, parse the XML, and integrate the new action into its memory structure.

3. The main database is stored in an inefficient XML format, so in order to startup, the iPhone version has to parse a 1/4 MB XML file, even though I have only 443 actions. If this were stored in an efficient text file, the data would only take up a small fraction of the size, and would be much quicker to parse.

4. The latency of the network connection becomes a serious issue when retrieving dozens of files over WebDAV. If there were a base file and a single list of deltas from that base which was appended to, each client could simply append to that file when making changes, and only read the new portion when updating a sync. WebDAV is sophisticated enough to allow partial reads of a file. This would make the typical sync process take much less than one second.

I really hope you guys implement a new sync system that is designed for both reliability and speed. Speed is critical to making the iPhone version work well, because I'm not using it to do a weekly review -- I want to look at a list or check off an item, or add an item to the inbox most of the time, and then close the app. Unfortunately since there are no background apps, I have to sit and wait until the program is done doing its thing, which means looking at a list is a 1-2 minute process, when it should be a 5 second process.

[QUOTE=kammeyer;70863]As for the architecture itself, I'm not sure where the biggest problems come from, but some aspects are obvious from looking at what drops on the webdav server.[/QUOTE]

With that kind of insight, maybe [I]you[/I] ought to be developing this app rather than the Omni folks.

-Dennis

[QUOTE=kammeyer;70863]
Telling users to contact support as if it is some problem with their configuration rather than a problem with the software itself is disingenuous.
[/quote]
For some users, it is an issue with their usage. For some users, it is an issue with bugs. For some users, it is an issue with their network, or the server they've chosen to use. In all of those cases, the Support Ninjas can help isolate the problem and guide the users to better results and practices, collect data needed to track down bugs, etc.
[quote]

1. The data files are too large and too complicated relative to what they are actually holding. Just adding an item to the inbox requires creating one or two XML files on the server, which contain tons of extraneous information. Just the file that contains the item contains all sorts of information about my computer. Here's the file generated from me adding a single two word action to the inbox:

<?xml version="1.0" encoding="utf-8" standalone="no"?>
<omnifocus xmlns="http://www.omnigroup.com/namespace/OmniFocus/v1" app-id="com.omnigroup.OmniFocus" app-version="77.41.6.0.121031" os-name="NSMACHOperatingSystem" os-version="10.6.2" machine-model="MacBookPro4,1"><task id="apXSog3buY1"><inbox/><added>2009-12-17T07:37:31.077Z</added><name>Test Action</name><rank>2080374784</rank><order>parallel</order></task></omnifocus>
[/quote]
Yes, it appears there's a lot of stuff there. And if you added another thousand words, it would grow by a thousand words. Most of the data that you apparently object to is there to ensure compatibility between clients of different versions and possible future new platforms. The rest of it describes where in the database that new action is supposed to live.
[quote]
2. The app creates a new XML file every time something changes, and then because the XML files are so verbose, it ZIPs them up. Thus, for every new action, the phone has to make an HTTP connection, download a ZIP file, unzip it, parse the XML, and integrate the new action into its memory structure.
[/quote]
Actually, it's a new XML file for every group of changes, though in normal operation the groups are usually of size 1.
[quote]
3. The main database is stored in an inefficient XML format, so in order to startup, the iPhone version has to parse a 1/4 MB XML file, even though I have only 443 actions. If this were stored in an efficient text file, the data would only take up a small fraction of the size, and would be much quicker to parse.
[/quote]
I do not believe that OmniFocus parses the entire XML database representation on startup unless the sqlite cache is out of date. It will obviously have to parse any new XML files that come in via sync if one is done at startup. My database currently has 4,554 actions and occupies 11 million bytes on the disk, and on my MacBook (which is much more powerful than the processor in any iPhone) it takes 35 seconds to rebuild the cache from scratch, which involves parsing the entire XML representation. It normally takes about 5 seconds to start.
[quote]
4. The latency of the network connection becomes a serious issue when retrieving dozens of files over WebDAV. If there were a base file and a single list of deltas from that base which was appended to, each client could simply append to that file when making changes, and only read the new portion when updating a sync. WebDAV is sophisticated enough to allow partial reads of a file. This would make the typical sync process take much less than one second.
[/quote]
Are you aware that OmniFocus supports syncing more than two devices, and without all changes being synced before another device makes a change? And that it has to be reliable in the face of network drops and very slow write times? Hmm, this is starting to look a bit more complicated...
[quote]
I really hope you guys implement a new sync system that is designed for both reliability and speed. Speed is critical to making the iPhone version work well, because I'm not using it to do a weekly review -- I want to look at a list or check off an item, or add an item to the inbox most of the time, and then close the app. Unfortunately since there are no background apps, I have to sit and wait until the program is done doing its thing, which means looking at a list is a 1-2 minute process, when it should be a 5 second process.[/QUOTE]
The current scheme is designed for both reliability and speed (in that order). There's a bug in the currently shipping version that results in inefficient processing of new data, but my suspicion without having seen the code is that with any reasonable abstraction, the same mistake would have been made had your proposal been in place. The version being tested now really flies -- I've got literally thousands of zip files that have yet to be compacted into my database because I've got a sync client that has been offline for almost a month now, and I can still add new items or check existing ones in just a few seconds.

If you've got a problem with slow startup on your iPhone, try turning off auto-sync if your usage pattern tends to be use one device for a while, then use the other device for a while. You can do a manual sync when you switch devices. When the new bits are deemed ready for public consumption, you can turn auto-sync back on with little penalty.