Synch Time on Iphone is totally unacceptable
 

Delete

It seems to have gotten worse in the last day or two.

It's taking me 2 minutes on a fast wi-fi, and probably 10-20 mins on EDGE (which doesn't really work for me, since my iPhone goes off after 2 mins).

I have 100 or so projects, but only 20 or so active ones; and most of those have 10 or less tasks.

I'm wondering if the database being synced contains items that have been checked off? I have 1000s of those, but don't know how to delete them.

Have a look at the OmniFocus.ofocus file on your MobileMe or WebDAV server -- select the file and right-click on it to get the contextual menu, then choose Show Package Contents. How many files are you seeing after you've synced? And how big is the whole thing?

You can use the File->Move Old Data to Archive... command to move your older completed items out of the active database, which should help speed things up. I think OmniFocus is able to read and write the individual transaction files without having to transfer/rewrite the whole package on each sync, so old stuff that isn't changing shouldn't be quite so much of a burden, but there's not a lot of memory or cpu horsepower on the iPhone, so archiving the old stuff is probably a good plan. One nice feature of the archive command is that it will tell you how many completed items there are before a given date which could be prospectively shuffled out to pasture.

I'm brand new to OF, so this is my first post. I have had horrible luck with iPhone syncing the last few days as well. The sync never finishes, regardless of connection type. The Synchronization area in Settings says only "sync currently in progress" and I don't ever see the progress bar any more. I have tried the Reset Database option on the iPhone, and have also reinstalled the app itself. OF on the Mac seems to be working just fine (v77.9.0.103638), as far as I can tell, since my two Mac copies are staying in sync, but this problem seems to have happened directly after one of the Sneaky Peek updates, not the iPhone 1.0.2 update. Maybe. Who can tell for sure? Sigh!

Relevant or irrelevant info: OF database sync file size is only 3.2MB. I have a fairly small collection of projects so far, so I know this is not a database size issue. I don't need to move old data to archive yet.

Just wanted to share that I'm having this issue too.

Me too, but not fixed
 

I was having this same problem over the last couple of days. I followed above instructions, opening the package (OmniFocus.ofocus) file on iDisk. There was 546 files there. I then moved old items to archive, setting the date to 8/1/2008. When I checked the OmniFocus.ofocus package again, the number of files dropped to just a handful. I then synced from my Macbook - selecting local copy when prompted. After this I opend OF on my iPhone and selected server copy when prompted. Everything is back to normal now.

I'll watch this closely. After finally feeling the relief of having control over my crazy project and task list - I was absolutely freaking out at the possibility of losing everything now that I'm trusting OF. Let's hope whatever this isn't lasting and get's fixed soon.


[QUOTE=lekowicz;43548]I'm brand new to OF, so this is my first post. I have had horrible luck with iPhone syncing the last few days as well. The sync never finishes, regardless of connection type. The Synchronization area in Settings says only "sync currently in progress" and I don't ever see the progress bar any more. I have tried the Reset Database option on the iPhone, and have also reinstalled the app itself. OF on the Mac seems to be working just fine (v77.9.0.103638), as far as I can tell, since my two Mac copies are staying in sync, but this problem seems to have happened directly after one of the Sneaky Peek updates, not the iPhone 1.0.2 update. Maybe. Who can tell for sure? Sigh!

Relevant or irrelevant info: OF database sync file size is only 3.2MB. I have a fairly small collection of projects so far, so I know this is not a database size issue. I don't need to move old data to archive yet.

Just wanted to share that I'm having this issue too.[/QUOTE]

My OmniFocus.ofocus file is at 2.2mb and contains 499 items. I delete all completed actions and projects religiously, and have 39 projects, each with about 4 actions on average, not one of them older than 10 days.

It is incredibly slow to sync on the iPhone even over wi-fi. Takes just a few seconds on the desktop.

It would be incredibly useful if Omni could let us know if the sync continues in the background if we quit OF? I believe quitting doesn't actually quit (holding the home button down does that), but am not sure. Now we have manual syncing it helps, but not if we have to sit and wait for a sync to finish before we can do anything else.

Mark

Now Unable to Synch
 

Same problem here: endless sync times. Checked OmniFocus.ofocus file on iDisk, cleaned out a ton of files, leaving those from this afternoon. Now I'm getting a "Unable to synchronize database with server" message, followed by "Unable to read document" and "No root transactions found in [url]https://idisk.me.com/[/url][username]/Documents/OmniFocus.ofocus/".

Can I just start from scratch, without losing my database?

I was going to try the archiving feature just to see what happens, but I can't do that because I have a couple shopping lists in OF, and archiving would remove all of those "completed" items. (I just started using OF less that a week ago, so my shopping items are about the oldest things in there.) The completed tasks for other projects number in the single digits, so not much to do away with there.

My package on MobileMe cloud disk is 2 MB, almost 500 files. It's taking over 5 mins to sync to iPhone with a fast, dependable wifi connection.

OK, now the sync has frozen iPhone OF. Had to end task on it.

Reopened iPhone OF. Loading is taking a looooonnnnnnng time.

If it would help for me to reduce my hundreds of projects to a dozen, please just tell us that, OmniGroup. This whole thing syncing problem is becoming unacceptable.

And OF's biggest competitor, Things, is about to release a sync version. I'm already in line.

Truly, I have a love/hate relationship with OF.

I can't believe this: iPhone OF has been "Loading..." the database for over 10 minutes on a fast wi-fi.

I guess if you make the mistake of ending task on the app while it's syncing, that completely corrupts the entire database, yes?, and you have to reload it from scratch, which I guess can take crazy long times.

Word to the wise.

Sorry, the slow performance is a bug. I haven't been able to reproduce this yet myself (my syncs are all still quite fast), but Brian did on Friday. Unfortunately, we didn't have time to get a full development environment set up with his phone before I had to leave on Friday, so I wasn't able to determine what was going wrong—just that something was.

I've been trying to reproduce the problem on my phone all day today, with no luck yet. As soon as Brian and I are back in the office tomorrow morning, I'm planning to finish getting the development environment set up for his phone so I can investigate further.

P.S. — Sorry I didn't chime in earlier, I haven't been scanning the forums for the last several days because I've been busy trying to track down this and other issues (mostly analyzing crash reports).

Thank you for the update Ken.

This isn't a problem I've been having, but knowing where things are at is always helpful when I am the one who's got the issue at hand going on.

Thanks,

Michael

[QUOTE=mhammitt;43613]Same problem here: endless sync times. Checked OmniFocus.ofocus file on iDisk, cleaned out a ton of files, leaving those from this afternoon. Now I'm getting a "Unable to synchronize database with server" message, followed by "Unable to read document" and "No root transactions found in [url]https://idisk.me.com/[/url][username]/Documents/OmniFocus.ofocus/".

Can I just start from scratch, without losing my database?[/QUOTE]
O.K. I simply trashed the whole OmniFocus.ofocus package from the iDisk, did a new OF sync, and we're back in business. Didn't lose anything, at least, and no more syncing problems.

[QUOTE=santra;43617]I guess if you make the mistake of ending task on the app while it's syncing, that completely corrupts the entire database, yes?, and you have to reload it from scratch, which I guess can take crazy long times.[/QUOTE]

I have not encountered this myself... I have done nothing in OF while syncing on iPhone, so my sync issues don't stem from this. Just an FYI.

Thanks for looking into it Ken! I'm going on a trip sans OF Mac, and have been worried that I'd end up losing any changes I make on OF iPhone while away. I'm sure it'll be all happy once I get back!

So what is happening when the double arrows on the phone are turning and what happens when it says Updating With Synced Data. In the first is it just locating the data on mobile me or is it locating and downloading? Is the second the result of the phone having downloaded the Database into a buffer and then writing it to the active Database?

A lot of work to manage
 

I think OF for Mac is absolutely fantastic but the sync times for iPhone render it unusable as an app. Certainly not enough to be a "trusted system". Like a lot of OF users, we've been waiting for it but it makes web-based systems like toodledo or Remember the Milk real possibilities now.

I don't know enough about WebDAV to compare it against other options but I can't help but feel that it isn't going to work for anyone who uses the desktop version extensively. It is so great at clipping and "pulling data in" to be a comprehensive application that it runs counter to what the iphone and AT&T network can handle.

At this point, I'm really frustrated with the iPhone app.

Forever sync workaround works!
 

I went to my webdav folder and renamed the OmniFocus.ofocus bundle to OmniFocus-backup.ofocus, and then synced OF on my mac. When that was done, I went to my iphone and synced from there, choosing the "From Server" option when prompted.

After that, the second syncs on both my mac and iphone took longer than they should, but did finish (mac took about 30 sec, iphone took about a minute or two). Since the 2nd syncs, the time they take to sync is less than 5 seconds for small amounts of change, on both devices. (This is with a db of 167 KB vs the old 831 KB version).

Whatever's going on, it has something to do with the syncing process, since that seems to be leaving the remote (iDisk/webdav) copy fragmented or something. Maybe the desktop app should be changed to occasionally do a complete re-upload.

We've tracked down the slow sync problem, and fixed it for the upcoming 1.0.3 release.

For the curious, the problem is triggered by a large number of transaction files: OmniFocus was trying to traverse the antecedent transactions for each transaction to figure out which transactions it needed to download from the server, and ended up in a Towers of Hanoi-like algorithm where every additional transaction made the process take nearly twice as long because we would revisit all the previous work yet again when looking at the new transaction.

The solution in 1.0.3 is to fix the algorithm to only look at each transaction once (which is how we meant for it to work in the first place), but a workaround you can apply now is to reduce the number of transactions by making sure each sync client connects on a regular basis (which lets OmniFocus clean up old transactions that it was only preserving in order to let out-of-date clients catch up). I guess that's why I hadn't seen this myself, yet; all my clients were connecting on a regular basis.

[While I wrote this, Ken posted the above post]

I have the same problems. Loading and syncing takes several minutes.

I had a look at my Mac's .ofocus file and noticed that it was about 3mb.
Some people at the forums thought this was really large so I ran the "CoalesceDatabase.scpt" script file supplied by Omni in one of those threads. It took the db size down to 1.2mb. No speed change in on the iPhone though.
Following this thread I checked the size and nr of files inside the .ofocus file on MobileMe and it was 1.7mb and contained 499 files while my iMac had 39 files. Something weird is definately going on with the nr of files here.

Ok so now I try the remove MM .ofocus file and resync with a fresh one from the iMac. There is still a lot more than 39 files in the online version compared to the stationary version. After several failed attemps and finder crashes (OSX does not like me to do file chores on my iDisk through finder, I had to use MM webclient at one point to delete the .ofocus file. Cloud version is now 1.1mb. Syncing is pretty much just as slow on iPhone.

Inside my iMac's .ofocus package there is one big file of 1mb. the rest of the 38 files are tiny. Whats that?

Even if I change nothing and resync on the iPhone it takes the same amount of time to sync. Is the sync not differentiental?

[QUOTE=Sandpiper;43649]So what is happening when the double arrows on the phone are turning and what happens when it says Updating With Synced Data. In the first is it just locating the data on mobile me or is it locating and downloading? Is the second the result of the phone having downloaded the Database into a buffer and then writing it to the active Database?[/QUOTE]

When the arrows are turning, OmniFocus is talking to the server. When it says "Updating with Synced Data" it's taking the data it downloaded and building a database out of it.

[QUOTE=colicoid;43751]
Inside my iMac's .ofocus package there is one big file of 1mb. the rest of the 38 files are tiny. Whats that?[/QUOTE]

That's all the past data that everyone agrees on, what remains after smooshing. The tiny files are new transactions that haven't been synced in by everyone (and subsequently smooshed). How many actions do you have in your file? You can find this in the terminal by doing

[code]
sqlite3 ~/Library/Caches/com.omnigroup.OmniFocus/OmniFocusDatabase2 'select count(*) from Task;'
[/code]

I've got 7165 actions in my db, which occupies 870 kb. I have very few files imbedded in the notes, but I would guess you might have some judging from the size of your file.

I wouldn't assume that iPhone sync time is necessarily only dependent on the time it takes to read or write that file; the iPhone isn't going to be much competition for your desktop machine in a cpu benchmark -- it's gonna get smooshed! Okay, I admit it, I like that technical term Lizard gave us :D

[QUOTE=Ken Case;43732]The solution in 1.0.3 is to fix the algorithm to only look at each transaction once (which is how we meant for it to work in the first place), but a workaround you can apply now is to reduce the number of transactions by making sure each sync client connects on a regular basis (which lets OmniFocus clean up old transactions that it was only preserving in order to let out-of-date clients catch up).[/QUOTE]

Even after resetting the database on my iPhone I still can't complete a sync. (I assume resetting the database is as up-to-date as you can get.) Also, I never saw this issue in OF 1.0.1. Was this code new in the 1.0.2 update, or was it there before but just didn't cause any problems?

Just installed the new iPhone firmware 2.0.1. Opened OF on iPhone, it started to sync, 5 minutes later something I've never seen:

the sync icon (double circular arrows) had an exclamation point in the middle. I tap on it and the error message says: "Unable to synchronize database with server. Timed out."

Groan.

Update: synced iPhone in iTunes twice. Tried again to sync iPhone OF over wifi--will not sync.

[QUOTE=Ken Case;43732]We've tracked down the slow sync problem, and fixed it for the upcoming 1.0.3 release.[/QUOTE]

Please--the sooner the better. I can no longer sync my iPhone after installing the new 2.0.1 iPhone firmware.

I either get the above error, or it takes 10-15 minutes EACH time to sync.

I give up. I'm reinstalling Things on my iPhone (and that app doesn't even sync yet!).

It's just That Upsetting--i.e., wanting to rely on software that I've paid for, and losing 1-2 hours a day now because it doesn't work. I know Ken has promised it will be corrected in 1.0.3, but that could be days or weeks--I can't afford to lose the reliability I place on PM/todo software on my iPhone now for that long a time.

The reason I get so enraged about this stuff is because I care so deeply and passionately about good software UI, reliability, accountability, and just plain integrity. What's ironic about this is that the iPhone release is "regular" software, and it's the version that's breaking down; whereas the desktop OF client I'm running is a beta (sneaky peak), and yet it's working fine. Great, in fact.

Would it be possible for OmniGroup to please Twitter when 1.0.3 comes out? I didn't even know when 1.0.2 was released...I only got the news from some obscure RSS feed; and then I had to dive deep into a forum to learn that it has to be "rebought" from iTunes Apps store to install correctly. An email announcement to all paying iPhone OF customers would be even better, of course, but a twitter is faster now (and less work for OG). Thanks.

Sorry to hear that things are not working for you. We are certainly working very hard to identify and fix all the issues that have been raised.

Note that we do typically twitter when updates are released. The twitter for 1.0.2 is here, for instance: [url]http://twitter.com/omnifocusiphone/statuses/873982450[/url]

As a follow-up, we have done some initial testing with iPhone OS 2.0.1 now, and we're not seeing any problems. I realize that doesn't help you, but I suspect that your increased problems are coincidental with the 2.0.1 release. Hopefully 1.0.3 with its performance improvements will solve the issue for you when it is released, which should be pretty soon if all goes well.

[QUOTE=whpalmer4;43759]That's all the past data that everyone agrees on, what remains after smooshing. The tiny files are new transactions that haven't been synced in by everyone (and subsequently smooshed). How many actions do you have in your file? You can find this in the terminal by doing

[code]
sqlite3 ~/Library/Caches/com.omnigroup.OmniFocus/OmniFocusDatabase2 'select count(*) from Task;'
[/code]

I've got 7165 actions in my db, which occupies 870 kb. I have very few files imbedded in the notes, but I would guess you might have some judging from the size of your file.

I wouldn't assume that iPhone sync time is necessarily only dependent on the time it takes to read or write that file; the iPhone isn't going to be much competition for your desktop machine in a cpu benchmark -- it's gonna get smooshed! Okay, I admit it, I like that technical term Lizard gave us :D[/QUOTE]

I had about 2.5k files.

I too was have problems and extremely long synch times with my iphone 3G. Last night I loaded all the updates to OF for the mac and iphone that were available. When starting up OF it recopied my database from my iMac to iDisk, and then I synched my iPhone, which reloaded it's database from iDisk. I then upgraded to 2.01.

Syncing is now very fast for both platforms. At one point via wifi on my iPhone it took 20 minutes, if it completed. Now it's a minute or two. I have around 70 projects and a few hundred actions.

We're moving in the right direction...looking forward to 1.03

[QUOTE=santra;43794]Please--the sooner the better. I can no longer sync my iPhone after installing the new 2.0.1 iPhone firmware.

I either get the above error, or it takes 10-15 minutes EACH time to sync.[/QUOTE]

I'm getting super-slow syncing like this too - 20 minutes is not unusual, and it's never less than 10. Syncing with the desktop client is fine - minute or two and it's done. I've been through every method to ameliorate the issue that's described here, but nothing's really helped.

So I'm looking forward to 1.03 also :)

fix for very slow/no sync of iphone to mobile me.

symptoms:
sync of omnifocus on the iphone (omni iphone) seemed to continue endlessly (spinning gears until phone slept or until switched apps)

Fix summary. (see below for more detail)
set iphone not to sync automatically (temp setting)
set iphone not to autolock (temp setting)
prepare omnifocus mac database
archived omni mac data completed items
rebuilt omni mac database
backed up then deleted idisk data
performed sync from omni mac
reset omni iphone database
re-configured omni iphone app for sync
synced omni iphone app
jumped up and down, chanted, and patted forehead three times...

Notes/assumptions:
sync of omnifocus on the mac (omni mac) seemed to work ok and generally took about a minute or so.
differences in iphone and mac data were slight and I could afford to lose latest iphone changes.
original OmniFocus.ofocus file size was 3.5mb on idisk
both mac and iphone on wifi with high speed internet connection.

set iphone not to sync automatically
iphone omnifocus app:settings:auto sync: off
(prevents undesired syncs)

set iphone not to autolock (temp setting)
settings:general:auto-lock: never
(gives time for sync to occur before iphone sleeps and locks)

prepare omnifocus mac database
viewed all completed items. unchecked those that I did not want to archive. to expedite, you can select multiple items in a project (cmd and/or shift click), control click on one of them, and uncheck "complete" from the contextual menu. (note: you may also want to flag these items to find them easily again in case you want to restore them to completed status later.)

archived omni mac data completed items
file:move old data to archive...
set date to yesterday. selected "Move to Archive"
(reduces data file size)

rebuilt omni mac database
file: rebuild database. "Rebuild"
(i suppose it might have been more appropriate to do this prior to archiving?)

backed up then deleted idisk data
quit omni mac (verify omni iphone not syncing)
mounted idisk
navigate to idisk:Documents:
selected all omni docs and did a finder copy to a folder on my mac. note: my idisk data seemed to have extra files besides the main data file which might have been remnants of earlier uncompleted syncs.
these were the items in my docs folder:

OmniFocus.ofocus
OmniFocus.ofocus-copy-in-progress-dKOHBli-8wB
OmniFocus.ofocus-copy-in-progress-fLgmnUCxDWe
OmniFocus.ofocus-copy-in-progress-lgHTJVJTC7-
OmniFocus.ofocus-copy-in-progress-oippbxpLfzl

after copying, attempted to finder delete these items from idisk. this process hung. (probably just an unrelated glitch)
rebooted mac, used safari to visit mobile me:idisk:documents and deleted all of the above omni docs via the web interface. it took two attempts to do this. the main data file remained after the first attempt, but was successfully deleted on the second.

performed sync from omni mac
launched omni mac and synced.
(verified that a new "OmniFocus.ofocus" file was created in idisk:documents)

reset omni iphone database
omni iphone: reset database
(this will delete your data on your iphone and require you to set up omnifocus syncing again)

re-configured omni iphone app for sync
(this is the same procedure originally used to set up iphone syncing)
omni iphone: settings: username
enter [email]username@me.com[/email] and dot me password
omni mac:preferences:sync:mobile me: share settings.
omni iphone: get settings from mac. complete dialog to set up omni iphone sync.

synced omni iphone app
omni iphone: click the sync icon (duh)

this worked for me. OmniFocus.ofocus file size decreased slightly to 2.7mb. omni mac sync time decreased slightly. omni phone sync time went from infinite to less than 30 seconds. at this point, after several syncs, none of the OmniFocus.ofocus-copy-in-progress-xxxxxxxx files have re-appeared.

you might want to reset the temporary settings changes in the first two steps above.
you might want to re-complete those items unchecked in step 3 above.

i'm sure that all of the previous steps and precautions were not necessary to fix the problem. if you discover an abbreviated procedure, please post.

jim

jmd,

Thanks for the detailed post, although I have to say that we're now entering Swift's "A Modest Proposal" territory :-)

Wow, It Worked
 

Thanks, JMD. That all worked brilliantly, and my iPhone syncing is now swift. I mean, in the real swift sense, not the Swift sense.

My only change was, since OF on my iPhone NEVER stopped syncing, I could not access the controls to turn auto syncing off. I just deleted then reinstalled the app.

The archiving worked well (unchecking, archiving, and re-checking my long reusable lists, which I hope not to have to do again!). Though the database is the exact same size now (3.2MB), [I]something[/I] fun happened in there to make it all work nicely.

Thanks to the Omni folks, too. Can't wait for 1.0.3, Ken.

Has this slow synch time been resolved or are we expected to go through a complete self diagnostic to cure our own issues.

I keep looking in App Store to see if there is a fix and nothing has come down.

If the sync is broken for a large number of people I don't see how this app can continue to be sold.

lekowicz,

I should add that, after following the steps, sync is now working for me, as well.

Supposedly OF Desktop was updated with a new sneaky peak to fix the ordering problem. My sync times are somewhat acceptable, but sync is not accurate. Change desktop, sync desktop, open iPhone, sync, and iPhone doesn't pick up changes. Have to do it a second time. So I'm seeing an accuracy problem, too.

I'm not willing to run jmd's fix. I have apps like Bookpedia that I paid less for that work flawlessly out of the box, so after going through hours and hours of grief over these sync issues, I'm reluctant to take the hour or so to run jmd's fix. And I have no guarantee or OG support on its continuing to work.

The recent fix in the sneaky peek will fix one direction, but you need the same fix on the iPhone app to fully fix it. 1.0.3 will hopefully be available soon.

jmd's fix is a more complicated rendition of a cleanup procedure we've also recommended: [url]http://forums.omnigroup.com/showthread.php?t=9273[/url]

I;ve just updated to the latest sneaky peak and run the coalesce script. It reduced my sync file from 2.2MB to 10KB!!!!

Now it syncs in seconds at both ends - well done Omni!

I'm curious about what your size will be in 24, 48 hours. I had the same success you mention, but then 2 days later I was back up to 200K. Now it's 3 days and I'm at 728K.

[QUOTE=ashleymills;44260]I've just updated to the latest sneaky peak and run the coalesce script. It reduced my sync file from 2.2MB to 10KB!!!!

Now it syncs in seconds at both ends - well done Omni![/QUOTE]

santra: The two ways your files grow are

(1) if you change the sync settings on your iPhone. Then you need to re-run the script.

(2) if you haven't synced a particular computer or iPhone recently. Then you need to sync that computer, wait an hour, sync it again, and your files should shrink on their own. If you will never sync that machine again -- because you don't own it anymore or whatever -- then you should run the coalesce script. (My database swells every weekend, and settles back down by lunchtime on Monday.)