Accepting applications to beta test the Omni Sync Server
 

We've been investigating what it would take to offer a server for customers to use to sync their OmniFocus data. (This would be an additional hosting option - none of the existing options would be going away!)

We've done some initial research, set a test server up, and would now like to start collecting data that will help us determine if this is something we can feasibly do long term. Specifically, we need to figure out what kind of resources (in terms of server hardware and data traffic) this would require.

If you'd be interested in such a service and are willing to store your OmniFocus sync database on our server, we'd appreciate it if you could head over to [URL="http://www.omnigroup.com/sync/"]http://www.omnigroup.com/sync/[/URL] and let us know your email address.

We plan to start sending out invites early next week, in the order that folks sign up. We'll be inviting folks gradually, with a goal of getting as many folks on the server as possible while maintaining acceptable performance.

Answers to a couple of questions I expect will be fairly common follow. Please post or [EMAIL="support@omnigroup.com"]email us[/EMAIL] if you have more questions!

[B]What's this cost?[/B]
Syncing OmniFocus through our sync server is free.

[B]Is this thing going to blow up and eat my data?[/B]
We'll make every effort to prevent that from happening, but it is a possibility. That said, if something does happen, nobody knows more about how OmniFocus sync works than the folks that run this server. :-)

[B]Will you guys look at my data?[/B]
We value your privacy. We'll only look at a customer's OmniFocus data when it's absolutely necessary to solve a problem, and we'll get their permission before doing so if possible. (If someone is misusing the server, that's a different matter.)

[B]Can I sync OmniGraffle or OmniGraphSketcher for iPad using this server?[/B]
Not at this time, but if that's something you'd be interested in, please [EMAIL="support@omnigroup.com"]let us know[/EMAIL].

Consider me 'Applied' As of now, I use MobileMe iDisk, but if there's anything I can possibly do to help improve OmniFocus; I'm there!

I've got a MacBook Pro and an iPhone, soon to be joined by an iPad. And I love OmniFocus!!

I will gladly help test this!
I have two macs, and ipod, and an iPad all of which need to stay in sync with OF. I use it every day.

Thanks!

would this be something where omnifocus would be accessible on the web as well?

i LOVE OF on my mac and iphone (and soon ipad...), but my biggest bummer is that my assistant isn't on a mac, and won't be anytime soon. i would switch from mobile me in a heartbeat and gladly pay for a different service if it had the ability for web-based input.

this probably is a completely different thing to do than what you're talking about, but i figured i'd ask as long as you're talking about using your own servers...

What is the point when there are so many other services that one can use to host the OF data?

Unless of course I would be able to access my data from a web browser. That would be cool and would open the market to those of us forced to used a lesser OS at work. You know which one I mean!!

[QUOTE=Dogsbreath;76019]What is the point when there are so many other services that one can use to host the OF data?

Unless of course I would be able to access my data from a web browser. That would be cool and would open the market to those of us forced to used a lesser OS at work. You know which one I mean!![/QUOTE]
They didn't say anything about creating a web application. If you read what is written, it's only a server to sync your data to.

I read what's written. I don't expect OG to tell us everything. Is OG just after a slice of the online disk space market, or is there more to this than initially meets the eye?

[QUOTE=Dogsbreath;76021]I read what's written. I don't expect OG to tell us everything. Is OG just after a slice of the online disk space market, or is there more to this than initially meets the eye?[/QUOTE]

We're not even remotely interested in the online disk space market. We'd just[*] like for all our customers to have a great syncing experience, and some have had trouble finding that elsewhere.

If you have a sync server that's already working well for you, though, there's no need to risk our beta server!

P.S. — I guess I shouldn't say "just": if offering this sync service does look to be feasible for the long term, we might explore offering other services as well.

I would love to be part of your beta. I use a MacBook Pro, iPhone and iMac.

This sounds great, I was about to move my OF data from MobileMe to mydrive.ch, but I guess I'll wait and test this first.

I wonder if one astute person may be able to pull data from the OF file and make a web front end for that data... Would be interesting to see if its feasible.

I know The Hit List uses sqlite3 for task storage, but their app is not nearly as good...

Just a thought.

Oh Please
 

Email entered.

I would love to get in on this beta. I'm a heavy OmniFocus user...even from back in the kinkless days. I'd love to help test your server idea.

My Macbook Pro, iPhone and occasional Mac Mini all sync together. And soon to be iPad as well.

is it going to be just WebDAV?
 

I was very excited on news that Omni was making their own sync server, but it looks like this is just another flavor of the same WebDAV syncing that OmniFocus is already using, no?

There's an awful lot of potential to add cool new functionality if my OmniFocus data lives in the cloud. A web app, perhaps? Or just faster syncing in general that doesn't require my iPhone to download and analyze the files.

Though on first glance, syncing is very fast. Maybe Omni's sync server really is doing some magic behind the scenes? :D

Additional services beyond the current functionality are certainly possible. The primary goal of this project is to gather information that helps us decide whether this is something we can do on a long-term basis.

Baby steps, in other words. :-)

It is great to hear that Omni offers an official sync option. Are there any advantages over the free SwissDisk at this point, speed, stability etc.?

Count me in. Very curious.

although there are other solutions out there, an out-of-box solution from OF would make it much easier for people to get going.

I would be willing to pay if it offered something more than sync , that's my +1 for a beautiful web-interface :)

Sync service is working well
 

I connected up my iMac, macbook pro, iPad (running the iPhone app), and iPhone to the new beta synch service. Seems much faster than using mobile me.

and slap a web app on top!
 

...and slap a web app on top! Seems like a logical next step. Would solve that "for some reason I have to work on a windows machine today" problem.

stoked. bring it!

Good work Omni, your sync server is very fast (and thanks for using SSL)!

I don't suppose you want to disclose your security (who can see what?) and backup details (is there a plan?) at all...?

Is it going to offer any other type of features beyond what MobileMe offers? Have you taken a look at the guys who set up Basecamp syncing with his own Webdav server? It is pretty cool.

BZ

Delighted to hear you're looking for alternative syncing options. I'm very interested in any new solution you can offer end users.

Good luck with it!

James

Dreaming awake...
 

Just wondering how difficult it would be to resuscitate the web-ui interface to plug it in on top of the webdav server... That would be a wonderful improvement for people working on windows/linux PCs...

cheers.

[QUOTE=intranation;76145]I don't suppose you want to disclose your security (who can see what?) and backup details (is there a plan?) at all...?[/QUOTE]

You can see your files. If you can see someone else's files, that's a bug. Our sysadmins can pull files off the sync server, but the rest of us can't. That's by design.

We've got redundant hardware in place, and we back up several times an hour. If something blows up, the same folks that run our online store will work on getting it back up and running ASAP.

We're not ready to make specific uptime promises yet, though. It is a beta, after all. :-)

Are there any plans to offer remote syncing with encrypted contents? I'd love to be able to sync my data somewhere centrally, but without ensuring the contents are encrypted for my eyes only, I can't trust it.

woohoo
 

I have signed up to offer whatever help I can. I am no OF heavy user, but I suppose the more the merrier.

I appreciate that these things take time and you are only looking at sync services for now, but I definite second a web app for managing OF data on-line. I have a macbook and an iphone and would love to bin my Windows PC at work...but I have more chance of winning the lottery than that ever happening.

Also, I am very interested in seeing what additional things could be possible over and above the basic sync services.

[QUOTE=curiousstranger;76593]Are there any plans to offer remote syncing with encrypted contents? I'd love to be able to sync my data somewhere centrally, but without ensuring the contents are encrypted for my eyes only, I can't trust it.[/QUOTE]

It's possible we'll add something like this down the line, but at the moment, the data is not encrypted on disk. I know there are some hosting providers out there that offer this sort of service, though...

[QUOTE=Brian;76652]It's possible we'll add something like this down the line, but at the moment, the data is not encrypted on disk. I know there are some hosting providers out there that offer this sort of service, though...[/QUOTE]

I'd love a pointer towards them if you can. Additionally to encrypted on disk, the keys should only be in my possession. Without some sort of custom storage access client, this would be difficult, so eventually it would have to be something implemented in Omnifocus I would think.

I've been using this since the moment it became available and I love it. Synching between devices has always been flakey if I wasn't at home on my home network, but this now works anyplace, anytime.

Could just be me, but synchronization seems quicker than it was through WebDAV on Dreamhost.

The current service is just WebDAV, as far as I can tell.

Is there any thought being given to have a sync application/service hosted by Omni that would handle the heavy lifting of synchronization, conflict resolution, etc., so the clients would be a little less loaded down by zip files and managing it individually?

I would love to test this, even though I do use MobileMe. It would really be great if Graffle, Outliner and Plan could also use an optional database format, with a "save as" or "export" option to create flat files, and sync the database between iMac and iPad as well... really, all you'd need to do is make a basic database with a blob field that just stores the flat file data; really, you could just add that as a "share on OmniServer" option for uploading and then use on the iPad... anyway... count me in!

I'd like to just say that syncing with the Omni Sync Server has been flawless.

Last year, I was hanging on the Mobile Me just for syncing Omnifocus, but I eventually decided it wasn't worth the cost. The I switched to WebDav for awhile, but was really hoping for Dropbox compatibility. Now I dont need either. Great implementation, guys!

Speed of sync?
 

I'm a new beta tester and just switched over from SwissDisk to the new Omni Sync Server.

I have a very small database of tasks and for some reason it seems to take a long time (a lot longer than SwissDisk) to synchronize on my iPhone 4 app.

(I will note that pushing the sync button on my Mac OF syncs near instantly.)

Am I alone? Thoughts?

Fireproof, depending on where you're located geographically, you may be getting a faster connection to the SwissDisk servers than ours.

(We're in Seattle; I'm not sure where the SD guys keep their servers. A forum visitor mentioned them being a Las Vegas-based organization in another thread, but that may just be a business office or something.)

Thanks Brian. I'm in Dallas and that could be it.

I'll continue to monitor it. Given that I can do other things while it's syncing, it hasn't been a show stopper yet. Just thought I would mention it.

Also, with the iPhone 4 and multi-tasking, it should continue to sync even if I get out of the application and go address an email or something, correct?

Some forms of network access appear to work in the background - apps can finish an upload they've already started, for example, or keep a streaming music connection going - but thus far, we haven't been able to get the level of network access OmniFocus needs to do sync in the background. We file the upload/download requests, but they get paused when another app comes forward.

We're going to take another look at this once OmniFocus for iPad is finished. Until then, sync just suspends itself, then resumes later.

I wish it would support sync to DropBox. I trust their Amazon S3 backend. Plus, you can get a free 2GB account.

I just signed up, looking forward to checking it out.
I'd love to see this support syncing of the entire Omni suite of apps. With all the complexity of syncing OF databases, it seems trivial in comparison to have OO and OG files synced so we can have one place for all our Omni needs. This is especially true as you roll out all the iPad and iPhone apps. You guys really need to offer one point of connection for all of these that can be accessed anywhere.

Very Slow Synchronization
 

[QUOTE=Fireproof;80483]I'm a new beta tester and just switched over from SwissDisk to the new Omni Sync Server.

I have a very small database of tasks and for some reason it seems to take a long time (a lot longer than SwissDisk) to synchronize on my iPhone 4 app.

(I will note that pushing the sync button on my Mac OF syncs near instantly.)

Am I alone? Thoughts?[/QUOTE]

Fireproof,

I am located in Toronto, ON, Canada, and I am experiencing same very slow sync issue. On my laptop the sync takes maybe 2 seconds. On my iPhone 3G, a direct sync to the OmniFocus on my MacBook used to take at most 15 seconds when there were lots of changes. Now, synchronization takes up to 10 minutes and sometimes I get a "server timed out" error.

Another interesting point: my database was 200kb in size before I switched over to Omni Sync Server. Now it is almost 400kb.

[QUOTE=MarcinK;83717]
Another interesting point: my database was 200kb in size before I switched over to Omni Sync Server. Now it is almost 400kb.[/QUOTE]

Interesting, I just performed another sync on my desktop and watched my OF db grow to almost 600kb and drop back to 200kb. I guess this is just normal synchronization behaviour

[QUOTE=MarcinK;83718]Interesting, I just performed another sync on my desktop and watched my OF db grow to almost 600kb and drop back to 200kb. I guess this is just normal synchronization behaviour[/QUOTE]

Yes, the apparent size of the database can fluctuate quite a bit, and so can the actual size of the database. Here are some reasons why:

Your database is stored as a base file, containing all the state that all the clients agree upon, plus a bunch of little zip files corresponding to changes made since that point, approximately 1 zip file per change made. When all the clients are in sync, the next client to sync will compact the database, moving all changes more than an hour old into that base file and eliminating the little zip files that held them.

Because the database is stored as a bunch of files in a directory with a flag set to tell the Finder to treat it as one file (a package file), and because the filesystem allocates space in blocks which are typically larger than needed to hold these small zip files, as you accumulate a number of them, the size of that package reported by the Finder will grow rapidly, although the actual number of bytes in the package file may not. This doesn't matter much as far as the disk is concerned, but it does if you are copying the file across the network, where you are happy that you don't have to copy as much as it might appear at first glance, especially if you are using a slow cellular connection. So, if you really want an accurate view of the size of your database, look at the number of bytes reported by the Finder in parentheses, not the number of KB/MB/GB initially listed, which can be misleadingly large as it represents the maximum amount of data that could be stored in the disk blocks occupied.

In addition to the apparent size fluctuations, there can be some actual size fluctuations due to syncing. If you attach a file to an action or a project, perhaps an audio note from the iPhone, or a picture, or embed a document, that is a change that will occupy one or more zip files. If you then go and delete that attachment, it will remain in your database (even though it will appear to have been removed) until all of the clients are synced again and the transaction(s) adding, editing, and finally deleting that attachment have been compacted into the "past history" portion of the database. It is sort of like putting a file in the Trash, but not immediately emptying it, rather relying on someone else to do so at an unknown point in the future.

Zip files can accumulate in relatively large numbers if you are very active, and especially if you have a device which doesn't get synced very often (thus delaying compaction). If the number continues to grow, check the Show Clients display in the Sync preferences panel in the Mac application to see if you have a client that isn't being synced. Sometimes if you reinstall the app you'll end up with a registered client that is no longer active.

I must be blind, but I can't see anywhere in the settings on my iPhone the choice of sync to the beta, like it is on the iPad. How do I get it to sync to my iPhone 4?

Never mind, I figured it out. Thanks! I think this is going to be my choice way to sync Omnifocus? I sure hope it can go through the Firewall at work! Then my two Macs, iPad and iPhone will all be in harmony! :D

Cloud Sync works nicely
 

I find that the cloud sync works very nicely. At the very beginning I had a couple very slow syncs until everything settled down, but since than it has been smooth sailing. I've set it up to sync automatically and most of the time I don't even think about it - it just works in the background.

Marcin

I just got my invite but have not actually switched over yet. I've been using WebDav on my own host and it works pretty well. However, having OmniFocus on a personal Mac, a work Mac, iPhone, and iPad makes synchronization a bit of an issue, especially since my database is pretty big and sync operations are a bit slow on the iOS devices.

Before I dive in with both feet, I was wondering if anyone has noticed a particular performance boost or decrease with the Omni server? I mean, aside from the couple of "first time I did it, it seemed slow" kinds of posts... have you seen an improvement by switching from your old method?

Gets me to wondering, if every item is in fact a little zip file in the OmniFocus directory, why you couldn't employ a kind of "trickle sync" and just sync up every updated item in a background thread after every change rather than batching them up. Quite often I'll be updating my daily checklists on the iPad when it suddenly starts "updating with synced data", which sometimes creates duplicates of the repeating items I just checked off. Are you guys thinking about moving away from the batch model?

Speeding up syncs
 

I can't speak to comparing other WebDAV syncs to Omni Cloud (which I think is WebDAV too) as I used to sync directly.

If you are looking at increasing speed of your sync, than removing all unnecessary 'attachments' works wonders. I do a lot of clipping of e-mails and webpages into OmnioFocus. The problem there is that all graphical content gets saved as attachments and significantly grows your OmniFocus database. I've cut my db down to 1/4 of its original size by removing all unwanted attachments.

In the Window menu there is an Attachment List, which makes removing attachments that much easier. Also, when I create new actions from e-mails, I'll explicitly select the portion of the e-mail that is of interest to me - this limits what gets clipped to what you selected.

That's a good suggestion. I hardly ever use any attachments, though, and they almost wind up on someday/maybe items that I never update. Worth checking to see if I left anything large tacked onto an item somewhere, though.

I can already sync for free ota using webdav, but if the price is within reason, I'd consider using the native solution. I already paid a premium for apps, but I guess it's totally worth the price after seeing what the competition on the app store offers (and lacks). There's one GTD service called Nozbe that charges like 10 dollars a month and I think that's way too much for syncing and the app and service is very subpar. It doesn't even do real GTD! If the pricing is closer to what Toodledo is offering (around 15 a year) I'd consider it as long as the apps get constant updates (something Things and Remember the Milk could never commit to).

Locking, un-stickying, and deprecating thread in favor of [URL="http://forums.omnigroup.com/showthread.php?t=18092"]this one[/URL]. Part of the updated info there is that we're not going to charge to host your sync data. :-)