Importing OPML (Outline Processor Markup Language) files
 

I'm trying to figure out if I can import an OPML file directly into OminFocus - at the moment I'm failing miserably.
I have a project management application (Merlin) that exports in OPML format (amongst others) and specifically mentions OmniOutliner as a target application for this file format. Since OmniOutliner is supposed to export to OmniFocus I was hoping that there's some way of getting an OMPL file directly into OmniFocus. I would not want to buy OmniOutliner just to use it as an intermediate step between the two applications.
When I try to import into OmniFocus the import dialogu doesn't even recognise the OMPL file as a valid import option. Anyone out there manged to complete this or something similar?
David.

You don't need to buy OmniOutliner to use it in read-only mode, and I think that's all you should need. Omni's policy is that you can use their document applications as readers for their files without charge -- helps promote their use if you can create a document with one and send it to someone who doesn't own the app!

Download OmniOutliner from the product page, fire it up and open the .OPML file you've got. Then select all the rows, copy and try to paste into OmniFocus. You'll get a dialog box that allows you to specify how OO columns map into OF data. There are some bugs right now (in OF 1.5, at least) which I've been writing up so it isn't quite trouble-free, but you should be able to get the actions/projects without too much trouble. The bugs I know about at the moment are that dates don't seem to transfer, and the cancel button on the dialog to map columns doesn't actually cancel.

Works a treat! Can't see any major problems that a small amount of editing won't solve and this'll save me an awful lot of retyping on something I'll only need to do occasionally.
Thanks.

I just came here to find out how I can transfer OPML to OmniFocus. I have OPML that comes out of CarbonFin Outliner on my iPod Touch (a nice, basic outliner). It's awesome that Omni has allowed read-only capability in their apps. This worked well.

I'll be glad to go to OmniOutliner on iPhoneOS if it ever comes out but for now CarbonFin is very useful.

[QUOTE=whpalmer4;50110]Download OmniOutliner from the product page, fire it up and open the .OPML file you've got. Then select all the rows, copy and try to paste into OmniFocus. You'll get a dialog box that allows you to specify how OO columns map into OF data[/QUOTE]

Simply dragging one or more OPML files onto a droplet app on your desktop would avoid the need for OmniOutliner and might give an easier workflow (and more control) if it needs to be done at all regularly.

I have placed a simplified version of the droplet which I use at:

[URL="http://www.complexpoint.macmate.me/Site/Import_OPML_into_OF.html"]http://www.complexpoint.macmate.me/Site/Import_OPML_into_OF.html[/URL]

It creates a date-stamped OF folder of the imported projects for each OPML file that has been dragged onto the droplet.

(This version just imports the text of each opml node, and any Omni-style _note attribute, but the applescript can easily be adapted to import any other columns/attributes which your opml may have.)

The draft code (to be saved in .app format) looks like this:

[CODE]-- DROPLET TO IMPORT OPML FILES INTO OMNIFOCUS

-- Ver 0.4 Now gives warning message if XML Tools osax
-- from http://www.latenightsw.com/freeware/XMLTools2/index.html
-- has not been copied to ~/Library/ScriptingAdditions


property pstrDialogTitle : "OPML to OmniFocus Ver 0.4"
property pstrImportFolder : "Imported from OPML"
property pEmpty : ""

-- TRY TO READ THE DROPPED FILE(S) AS OPML
-- AND TRANSLATE CONTENTS OF EACH FILE
-- TO A NEW DATE-STAMPED FOLDER OF OF PROJECTS
on open lstfDropped
set {lstFolders, lngProjects, lngFiles} to {{}, 0, 0}
repeat with fDropped in lstfDropped
if IsOPML(fDropped) then
set strFile to ReadTextFile(fDropped)
try
set xmlDoc to parse XML strFile
set xmlContent to (a reference to XML contents of (last item of XML contents of xmlDoc))
set lstActivities to XML2List(xmlContent)
set {lngActs, oFolder} to PlaceInOF(lstActivities)
set end of lstFolders to oFolder
set lngProjects to lngProjects + lngActs
set lngFiles to lngFiles + 1
on error
set the clipboard to "http://www.latenightsw.com/freeware/XMLTools2/index.html"
set strError to "
This script requires installation of Late Night Software's free XML Tools osax

From:

http://www.latenightsw.com/freeware/XMLTools2/index.html

(This URL is now in your clipboard, and can be pasted to a browser)

The version of the XML Tools.osax which matches your operating system should be copied to a folder with the path:

~/Library/ScriptingAdditions
"
display dialog strError buttons {"OK"} with title pstrDialogTitle
tell application id "com.apple.finder"
set strAddns to "ScriptingAdditions"
set fldrLibrary to folder ((path to home folder as string) & "Library") as alias
if not (exists folder strAddns of fldrLibrary) then
set fldrAddns to (make new folder at fldrLibrary with properties {name:strAddns})
else
set fldrAddns to folder strAddns of fldrLibrary
end if
open fldrAddns
activate front window
end tell
return
end try
else
display dialog "Expected file with extension .opml" with title pstrDialogTitle
end if
end repeat
ShowResult(lngProjects, lstFolders, lngFiles)
end open

-- Give feedback to user
on ShowResult(lngProjects, lstFolders, lngFiles)
if lngProjects > 0 then
set strMsg to ((lngProjects as text) & " projects placed in OmniFocus" & return & return & ¬
"(" & lngFiles as text) & " opml file"
if lngFiles > 1 then
set strMsg to strMsg & "s imported)"
else
set strMsg to strMsg & " imported)"
end if
display dialog strMsg with title pstrDialogTitle
tell application id "com.omnigroup.OmniFocus"
tell front document
set recProps to {selected view mode identifier:"project", focus:lstFolders}
set oWin to make new document window with properties recProps at front of document windows
end tell
activate
end tell
else
display dialog "No projects found in " & strFile with title pstrDialogTitle
end if
end ShowResult


-- CREATE A FOLDER OF PROJECTS IMPORTED FROM OPML
on PlaceInOF(lstActivities)
set lngActivities to length of lstActivities
if lngActivities > 0 then
tell application id "com.omnigroup.OmniFocus"
tell front document
set strName to pstrImportFolder & " " & (current date)
set oFolder to make new folder with properties {name:strName}
my FillOFFolder(oFolder, lstActivities)
end tell
end tell
{lngActivities, oFolder}
else
{0, missing value}
end if

end PlaceInOF

-- Transfer activities to the specified project (or parent task)
on FillOFFolder(oFolder, lstActivities)
using terms from application "OmniFocus"
tell oFolder
repeat with oAct in lstActivities
set {strName, lstChiln, strNote} to oAct
if strNote is missing value then
set oProj to make new project with properties {name:strName}
else
set oProj to make new project with properties {name:strName, note:strNote}
end if
if length of lstChiln > 0 then my FillOFProj(oProj, lstChiln)
end repeat
end tell
end using terms from
end FillOFFolder

-- Transfer activities to the specified project (or parent task)
on FillOFProj(oProj, lstActivities)
using terms from application "OmniFocus"
tell oProj
repeat with oAct in lstActivities
set {strName, lstChiln, strNote} to oAct
if strNote is missing value then
set oTask to make new task with properties {name:strName}
else
set oTask to make new task with properties {name:strName, note:strNote}
end if
if length of lstChiln > 0 then
my FillOFProj(oTask, lstChiln)
end if
end repeat
end tell
end using terms from
end FillOFProj

-- READ FROM PARSED XML TO SIMPLE APPLESCRIPT LIST WITH TITLE SUB_LIST AND NOTE
on XML2List(lstContent)
set lstText to {}
repeat with xmlNode in lstContent
try
-- GET TEXT OF NODE
set recAttribs to (a reference to XML attributes of xmlNode)
set lngAttribs to length of recAttribs
if lngAttribs > 0 then
set end of lstText to {|text| of recAttribs}
else
set end of lstText to {pEmpty}
end if

-- GET ANY CHILDREN OF NODE
set refNewNode to (a reference to the last item of lstText)
set lstChildren to (a reference to XML contents of xmlNode)
if length of lstChildren > 0 then
set end of refNewNode to XML2List(lstChildren)
else
set end of refNewNode to {}
end if

-- GET ANY OMNI-STYLE _note OF NODE
if lngAttribs > 1 then
set strNote to pEmpty
set strNote to _note of recAttribs
if length of strNote > 0 then
set end of refNewNode to strNote
else
set end of refNewNode to pEmpty
end if
else
set end of refNewNode to pEmpty
end if
end try
end repeat
lstText
end XML2List


on ReadTextFile(fFile)
open for access fFile
try
set strContents to (read fFile)
on error
set strContents to ""
end try
close access fFile
return strContents
end ReadTextFile

-- Check that the file has an .opml extension
on IsOPML(fDropped)
set strFileName to fDropped as string
set {strDelim, text item delimiters} to {text item delimiters, "."}

set lstParts to text items of strFileName
set lngParts to length of lstParts
if lngParts < 2 then
set blnIsOPML to false
else
set strSuffix to (item lngParts of lstParts)
ignoring case
set blnIsOPML to (strSuffix is "opml")
end ignoring
end if
set text item delimiters to strDelim
blnIsOPML
end IsOPML

[/CODE]

I have updated the OPML2OF.app file at [URL="http://bit.ly/djsGUF"]http://www.complexpoint.macmate.me/Site/Import_OPML_into_OF.html[/URL] so that it now warns the user if the free [URL="http://www.latenightsw.com/freeware/XMLTools2/index.html"]XML Tools osax[/URL] from Late Night Software (which this script requires) has not yet been copied to ~/Library/ScriptingAdditions.

Rob.

[COLOR="White"]--[/COLOR]

Hi,
I've installed the XML Tools.osax and I downloaded the OPML2OF.app. But when I dorp an OPML file on the App I always get the error message that I have to install the XML Tools. Any solutions?

I'm on 10.6.6

[QUOTE=Ilajn;93688]I've installed the XML Tools.osax[/QUOTE]

What happens if you run this script:

[CODE]on run
display dialog XML comment as string
end run[/CODE]

or run the following line in a Terminal.app window:

[CODE]ls ~/Library/ScriptingAdditions[/CODE]

?

With the goal of getting my brainstormed nodes out of MindNode Pro into OmniFocus, I happened on your post about your applet, installed it along with XML Tools component and HOCUS POCUS, the OPML file exported from MindNode Pro was imported flawlessly into OmniFocus. Thanks for your work! It's very much appreciated!

[QUOTE=RobTrew;93689]What happens if you run this script:

[CODE]on run
display dialog XML comment as string
end run[/CODE][/QUOTE]

A window pops up saying "XML comment".

[QUOTE]

or run the following line in a Terminal.app window:

[CODE]ls ~/Library/ScriptingAdditions[/CODE]

?[/QUOTE]

The Terminal.app puts out:
"XML Tools.osax"

Ok, the next things that I would check:[LIST=1][*]That the version of the .osax the right one for your version of the operating system.[*]The provenance of the opml files (there may be something about their content that the script is choking on).[/LIST]
Finally, you could try this version (attached as zip), and see where the error occurs.

[QUOTE=RobTrew;93779]Ok, the next things that I would check:

That the version of the .osax the right one for your version of the operating system.
[/QUOTE]
It's 2.9.4 for OS X 10.6.
[QUOTE]
The provenance of the opml files (there may be something about their content that the script is choking on).
[/QUOTE]
Well, if I had this knowledge I would write scripts on my own... May I send you a sample?

[QUOTE]
Finally, you could try this version (attached as zip), and see where the error occurs.[/QUOTE]

Output: application id "com.omnigroup.OmniFocus" of «script» kann nicht gelesen werden. (canot be read)

[QUOTE=Ilajn;93839]Output: application id "com.omnigroup.OmniFocus" of «script» kann nicht gelesen werden. (canot be read)[/QUOTE]

I assume that you do have OmniFocus open and running when you run the script ?

(You can upload a zipped .opml sample to this thread if you like)

[QUOTE=RobTrew;93841]I assume that you do have OmniFocus open and running when you run the script ?

(You can upload a zipped .opml sample to this thread if you like)[/QUOTE]

Yes, OmniFocus is open and running while using either the droplet or the testscript.

Sample follows later on, I'm busy at the moment.

[QUOTE=RobTrew;93841]

(You can upload a zipped .opml sample to this thread if you like)[/QUOTE]

Here is the sample:

[QUOTE=Ilajn;94016]Here is the sample:[/QUOTE]

Trying to open this file as OPML with OmniOutliner3 trips the error message:
[INDENT][I]The document "Example English.opml" does not appear to be a valid OPML (Outline Processor Markup Language) file. Please check its contents. If you believe this file to be valid, please file a bug.[/I][/INDENT]
You may need to initiate a conversation between the authors of Projekt (which appears to have generated this file) and the engineers at Omni - some divergence here in what constitutes OPML, I think ...

PS

There seem to be characters in the linkId0 attributes which are not XML-compliant.

[IMG]http://farm6.static.flickr.com/5017/5470900970_2eeb2757c5_b.jpg[/IMG]

If Projekt can be adjusted to write these out correctly, you should be fine. Until then, you would have to hand-edit the raw text to remove the link ids (e.g. in your sample, from the Meetings, To-Dos and Memos lines ...)

[COLOR="White"]--[/COLOR]

PPS

You should be able to strip out the illegal character sequences (assuming that they are only in LinkId attributes) by running the file through a search and replace shell command before sending it to any kind of XML processor, whether the XMLTools.osax, OmniOutliner, or any other tool which expects compliant XML.

e.g. in a Terminal.app window something like:

[CODE]sed 's/linkId0="[^"]*"//g' ~/Desktop/Example\ English.opml > ~/Desktop/mended.opml
osascript ~/Desktop/OPML2OF.app ~/Desktop/mended.opml[/CODE]

(Replacing the reference to your desktop with the paths of the folders containing the script and OPML file(s))

(Note that calling the script on the command line with an argument, using osascript, only works with version [URL="http://www.complexpoint.macmate.me/Site/Import_OPML_into_OF.html"]updated today)[/URL].

[COLOR="White"]--[/COLOR]

I have also updated [URL="http://www.complexpoint.macmate.me/Site/Import_OPML_into_OF.html"]OPML2OF[/URL] to improve handling of OPML nodes which do not have the [I]_Note[/I] attribute used by Omni products.

[QUOTE=RobTrew;94046]I have also updated [URL="http://bit.ly/djsGUF"]OPML2OF[/URL] ... [/QUOTE]

Hi, I can't unzip this file. "Fehler 1 - der Vorgang ist nicht zugelassen." which means something like: Error 1 - this action is not allowed.

I'm in contact with the developer of Project. He wrote:

[QUOTE]
I see the problem. The LinkIds are internal adresses for the agenda entries. These adresses are produced by your phone (for entries created from within the calendar application) or by Projekt (for entries created from whithin Projekt). I had never seen entries formed like the ones I can see on your screenshot. What is the model of your phone ?
Envoyer
[/QUOTE]

[QUOTE=Ilajn;94200]Hi, I can't unzip this file. "Fehler 1 - der Vorgang ist nicht zugelassen." which means something like: Error 1 - this action is not allowed.[/QUOTE]

I have refreshed [URL="http://bit.ly/djsGUF"]the file[/URL]. (If there still seems to be a difficulty I will put it somewhere else).

[COLOR="White"]--[/COLOR]

[QUOTE=RobTrew;94201]I have refreshed the file. (If there still seems to be a difficulty I will put it somewhere else).[/QUOTE]

Extracts and runs here, at least. Interesting that the old zip file was 1 MB and the new one not even 1/10 of that!

[QUOTE=RobTrew;94201](If there still seems to be a difficulty I will put it somewhere else).

[COLOR="White"]--[/COLOR][/QUOTE]

No poblem: It extracts, now. Thank you.

Hello,

I'm back because things didn't improve. Another application on my phone may cause the problems with the messed up linkIds.

I tried OPML2OF on a document without links and linkIds. It still ends up in the former mentioned errors. I wonder wether it might be a problem that I didn't buy Omnifocus directly at Omni but in the App-Store (yes, it is the OS X application, I didn't install an iPhone/Pod/Pad version).

For testing I attach an example OPML file that doesn't contain linkIds.

You might be right about that — Rob's script had some instances where it referred to app id com.omnigroup.OmniFocus and that isn't correct if you have the OmniFocus from the App Store.

I made a version of the script that uses the other form of address. You can download it and try it out to see if it works any better: [url]http://dl.dropbox.com/u/971671/OPML2OF.zip[/url]

Let us know if that makes it work or not.

As a postscript, I have now amended the file to [URL="http://www.complexpoint.macmate.me/Site/Import_OPML_into_OF.html"]ver 0.6[/URL] which uses the form:

[CODE]tell application id "OFOC" -- OmniFocus[/CODE]

which should, I think, be compatible with Appstore-purchased and directly-purchased versions of OmniFocus, and perhaps even (touches wood) OmniFocus 2, when it comes ...

If Omni are using variant bundle-identifiers for all their AppStore versions, then there may be quite a lot of these edits to make. Apple was arguing when OS X 10.5 was released that the switch to the [I]id + bundle identifier[/I] form would be a good idea because developers were less likely to change the bundle name than the application name. This seemed to make sense:
[CODE]tell application "OmniGraffle Professional 5"[/CODE] for example, did look as if it might have a shorter shelf-life than:
[CODE]tell application id "com.omnigroup.OmniGrafflePro"[/CODE]
but 天有不测风云 (something like: 'the extreme sensitivity of non-linear systems to initial conditions makes them hard to model and predict' or just 'winds and clouds are hard to fathom' - though Google Translate offers a more extreme summary) and in retrospect it might have been more prudent to retreat to:
[CODE]tell application id "OGfl" -- OmniGraffle
tell application id "OFOC" -- OmniFocus
tell application id "OOut" -- OmniOutliner
tell application id "OPla" -- OmniPlan[/CODE]
etc, but this seemed a bit illegible at the time ...

[COLOR="White"]--[/COLOR]

[QUOTE=whpalmer4;94643]

Let us know if that makes it work or not.[/QUOTE]

Hooray! It works - unless the umlauts didn't import correctly. The source OPML file is in UTF-8 and e.g. TextEdit shows the umlauts correctly. I didn't find an option within OmniFocus to make it aware of UTF-8. May be I can use some text editor to work around this. But the less steps to perform the better it is. Is there a possibility to get the umlauts directly into OF?

It's the same with Rob's script OPML2OF ver 0.6.

[QUOTE=Ilajn;94665]Hooray! It works - unless the umlauts didn't import correctly. The source OPML file is in UTF-8 [/QUOTE]

Does the header of the OPML file specify UTF-8 ?

If not, it might be worth experimenting, in the script file, with changing the parse XML line, adding [I]encoding "UTF-8"[/I] at the end.

[QUOTE=RobTrew;94674]Does the header of the OPML file specify UTF-8 ?
[/QUOTE]

Is this first line of the file correct?

[CODE]<?xml version="1.0" encoding="UTF-8"?>[/CODE]

With this line I don't get umlauts :-(

[QUOTE=Ilajn;94680][CODE]<?xml version="1.0" encoding="UTF-8"?>[/CODE][/QUOTE]

That looks fine, and I'm getting a problem with Chinese too, when I import OPML generated by OmniOutliner into OF.

My first guess that it has something to do with XML Tools, but the bottom line is that I haven't understood it yet. I know rather less about text encoding than I should.

I'll give it some thought ...

I worked around the loss of umlauts when importing an UTF-8 encoded OPML file into OF. Here is the way although I don't understand why it works:

I open the original OPML file which comes from my phone in TextEdit. I don't change anything in the file except that I save it with western european Mac-Roman encoding. Importing this OPML file using OPML2OF I get all my umlauts. But this will surely not work for chinese:-(

I checked what happens if I try some other text encoding options in TextEdit:

Using[LIST][*]UTF-16 OPML2OF ends up in saying that I have to install the XML Tools first.[*]UTF-8 doesn't change anything.[*]Western European Mac-Roman see above[*]Western European Windows Latin 1 I get other symbols than using UTF-8, but I don't get umlauts[/LIST]

The culprit seems to be Applescript's built-in [I]read[/I] function, which still appears to be reading text as MacRoman.

I have now rewritten ReadTextFile(), dropping [I]read[/I] in favour of a very simple use of [I]cat[/I] in the shell:

[CODE]on ReadTextFile(fFile)
do shell script "cat " & quoted form of (POSIX path of fFile)
end ReadTextFile[/CODE]

On my system this fixes the output for the umlaut (and for Chinese characters too).

I have updated the file for [URL="http://www.complexpoint.macmate.me/Site/Import_OPML_into_OF.html"]download[/URL].

Dear Rob,

thank you so much. It finally works and I'm happy :-)

Just one more request for improvement: Would it be possible that your script asks the user wether to make the "filename" a project or the first level entries of the file?

[QUOTE=Ilajn;94732]Would it be possible that your script asks the user whether to make the "filename" a project or the first level entries of the file?[/QUOTE]

Well, yes,

天下无难事 有志者成之
天下无易事 有恒者得之

I think it would be possible.

i.e. most things will yield, in the end, to the force of will, but it helps to be persistent ...

I probably won't get to that for a while, I'm afraid, but the source code is there, so anyone is welcome to experiment ...

Note: OSX Lion hides the user's library folder. In order to install the the XML Tools.osax file you will need to make the library folder visible again.

Run this in terminal. One way to get to terminal is to click the spotlight icon in the top right hand corner of the screen and then type terminal.

[QUOTE]chflags nohidden ~/Library/[/QUOTE]

if you are worried about damaging the folder if it stays visible (Apple must have hidden it for a purpose - right) then you can hide it again by typing this into terminal.

[QUOTE]chflags hidden ~/Library[/QUOTE]

Great script, very useful, worked first time for me. It will be very helpful in planning projects in iThoughtsHD for import into OF.

Thanks you for creating and sharing the script.

Thanks philrob for that terminal command. I know you can turn on system files in the view menu to show the Library file but then you also see all the other system files (which I do not want to see). This makes things much cleaner.

Thanks a lot RobTrew for your work on this.
In my case it makes it really easy to use OmniFocus and MindNode together.

I knew about your Web site yet and this one seems not to be featured there. It really should IMO !

Regards

Added link
 

[QUOTE=bzhgeek;107754]I knew about your Web site yet and this one seems not to be featured there. It really should IMO [/QUOTE]

Thanks ! Very good to know that it is proving useful.

As for that web-site, you have reminded me that I will need to decide what to do with its contents ...

(Apple, in its new guise as a [URL="http://www.forbes.com/sites/steveschaefer/2012/01/25/blowout-quarter-pushes-apple-over-exxon-in-market-cap/"]$400bn[/URL] concentration of capital, a sort of vacuum cleaner in the sky, to adapt [URL="http://www.nybooks.com/articles/archives/2012/feb/23/were-more-unequal-you-think/?pagination=false"]an image used more generally[/URL] by Andrew Hacker in the NYRB, seems to have decided that web-hosting is inconsistent with the maximisation of share-holder value ...

They will pull the plug in June, I think).

[COLOR="White"]--[/COLOR]

Oh please keep it online !

Notes come into OF as 'block', returns lost
 

First, thanks to all, especially RobTrew, for all the very useful work!

When I use OPML2OF on OPML files, all works fine except that any notes attached to projects or actions lose any 'internal returns' in the text, so that after running the droplet the text of the note appears in OF as a single 'block'.

When I open the exported OPML file in OmniOutliner to check before dropping the file on OPML2OF, the note appears correctly, with returns breaking it up into paragraphs as in the original, but those returns somehow 'get lost in translation'.

I've tried tweaking a couple of things in the script, but no luck. I did try going back to version 0.5.1 to see if it had anything to do with the change from the AppleScript 'read' command to the 'do shell script cat …', but the result is the same with both 0.5.1 and 0.6

Any ideas?

Typical. I find that I need to do this, search for a way, find this thread, and then find I am too late because mobile me has closed...

You're not too late, you just need to go somewhere else. Rob has transplanted the entire tree to [url]http://www.complexpoint.macmate.me[/url].

[QUOTE=whpalmer4;112715]Rob has transplanted the entire tree to [url]http://www.complexpoint.macmate.me[/url].[/QUOTE]

… to which I have added a page at:

[URL="http://www.complexpoint.macmate.me/Site/Import_OPML_into_OF.html"]http://www.complexpoint.macmate.me/Site/Import_OPML_into_OF.html[/URL]

[COLOR="White"]--[/COLOR]