I made a HAT

I'm not talking about the kind of hat you wear; I'm talking about a Help Authoring Tool or H.A.T. Odd kind of thing for a technical author to do, but then again, I'm an odd kind of technical author.

I'd originally wanted to be some kind of scientist, inspired by the likes of Feynman and Hawking, but after a year's worth of degree physics, the only thing I'd learned was that I wanted to do something else. I was too messy to be an experimental physicist and I lacked the intuitive mathematical ability to be a theoretician. But during my year at Leeds, I'd joined the computer society and had been taught BASIC by one of the members, who I seem to recall was called Mike Ford. I'd then started reading computer magazines such as Personal Computer World and Byte and started looking longingly at the home computers that were appearing in the windows of the local Tandy store.

When I left my physics degree, I was desperate to find something to do, and since I'd been awarded a three year maintenance grant, something which must sound like an incredibly luxury to modern students, I applied for a HND, which was a two-year course that was equivalent to a pass degree. I wasn't confident enough to sign up for a dedicated computer course, so I ended up taking a HND in Mathematics, Statistics and Computing. Suffice it to say, it's only the computing knowledge that I've retained. A few years later, I'd managed to get another year's worth of maintenance grant to attend a Polytechnic Diploma course in Computer studies at Hatfield Polytechnic, which would raise my computer qualifications up to approximately Honours degree level. As part of that course, I found myself writing a compiler for a language called FORTH, creating the code by reverse engineering the manual from a Jupiter Ace that I'd bought during my years off. I also wrote what has to qualify as my first ever computer manual, which was a fifty page explanation of how my compiler worked that I'd created using WordStar. I remember going to a Science Fiction group in London, attending with some of the members of the Hatfield SF society. I was asked what I wanted to do, and I responded with "Technical Journalist". The truth was, I wanted to be Guy Kewney, as I was fascinated by computers and I had a tonne of opinions that I was eager to share. If I'd had Guy Kewney's talent, I might have succeeded.

Twenty years later, I was working as a full time technical author. I'd written two manuals, a couple of books, and I'd even managed to get my work published in computer magazines.  I'd spent my career as a writer but I'd never lost my interest in computers and I'd never lost my programming skills. For the moment, I was producing help files and I was having problems. I'd created my first help files as a freelancer using a package called HDK, a process which took a series of documents in Microsoft Word and converted them into Windows '95 help. The only snag, was that Word documents are notoriously finicky and even a slight problem with the original source formatting could make the conversion process fail. If that happened at the last minute, which it invariably did, I was forced to  frantically check through the sources to find the heading or image that had caused the problem, and although I always managed to make the fixes and produce the help files, I hadn't enjoyed the process at all.

I'd joined my current employer as a tester/documenter and I'd quickly discovered that the company had a bunch of printed manuals that they were eager to convert into help. Since I'd done that before, I volunteered my services, but rather than choose HDK and have to fight with the formatting again, I recommended that they chose the leading Help Authoring Tool at the time, which was Doc-to-Help. After a proof of concept that was shown to a major customer, and which got good feedback,  I was tasked with converting the other documents. As I was no longer doing any testing work, my job title was changed to "Technical Author".  When it was later decided to make the help context-sensitive, I implemented this using my own macros which I created using Visual Basic, something that I found easy because of my computer skills. I've have preferred to have used the out-of-the-box functionality from Doc-to-Help, but I couldn't do that because our software wasn't compatible. The process of creating the help was still a bit clunky and it usually took around thirty minutes to perform the compilation, but it generally worked. When it failed, I swore under my breath and searched for floating images in the source documents, which was usually the cause of any problems.

Everything went fine for a number of years, until the point the we were forced to move to HTML Help. It made sense to move to another tool, because we didn't need the single source approach used by Doc-To-Help, and it made no sense to labourously convert our documents from Microsoft Word into HTML when we could just as easily write them in that direct. We choose Robohelp and it was a good choice. It might have been hard to implement our quirky context-sensitive help system on that platform, and it might have gone wrong occasionally, but it still served us well.

There were only two problems:

The first was that Robohelp required Microsoft Word in order to operate, and it only worked with the latest version or earlier. This meant that whenever our company updated to a new version of Office, we'd have to buy a new version of Robohelp as well, something which I'm sure provided an invaluable revenue stream for the suppliers.

The other problem was that we were going to produce a cut-down version of our property management software that would need to include a simplified version of the help files. It didn't make sense to have separate source documents for each version as most of the information would be the same, and every change we made to the software would need documenting twice. But at the time, Robohelp didn't support conditional compilation, so the only alternative would be to replace it with something that did. Unfortunately, there was nothing else on the market that suited our needs precisely.

After discussing the issue with my bosses, I therefore created my own help development tool, figuring that the cost involved in development would be less than the cost involved in purchasing a commercial package when it became available and having to update that system on a regular basis.  The whole idea might seem utterly nuts, as Robohelp was created by dozens of programmers, and I was just a technical author who was still having to spend a lot of my time producing the existing help file. I did, however, have a massive advantage, because the only things I needed was something that would create HTML help files, support conditional compilation and implement our context-sensitive help system. I hadn't been using ninety percent of Robohelp's functionality anyway, so I wouldn't need to replicate it.

Another thing in my favour, was that the actual help compilation was being performed by a freely available tool called the HTML Help Workshop. The sources for this tool were the HTML files used to provide the help content and a few XHTML files used to generate the index and the table of contents information. It would technically have been possible to create our entire help system using the HTML Help Workshop, but that would have been incredibly difficult and time consuming, which was the reason we'd originally bought Robohelp. But HTML editors were widely available and there were plenty of free ones that would potentially serve our purpose. The only things my HAT had to do, would be to create the required project files, deal with the conditional compilation and pass the results to the compiler.

As I'd been badly scarred by my experiences with using dedicated project files, I decided to store all the information required to create the help in the HTML source files, using a similar system to that which Microsoft had adopted for the original Windows 95 help. The choice of programming language required something that was freely available and which was extremely efficient at stripping information out of text files. I might have chosen Perl, but I decided at the time that Python was a better fit. The big downside of either of these languages was that they were both very bad at creating user interfaces, and indeed, that would turn out to be the hardest part of the project.

After about six months work, I ended up with a tool that I called Python Help. The index and table of contents information were stored in META tags in the individual source files and I marked out the text for the conditional compilation using standard DIV and SPAN tags in combination with some appropriate CSS classes. I could have simply hidden the unnecessary text using CSS, but I quickly discovered that the hidden text would still be available from the compiled help's search system, which would have confused our customers enormously. I therefore created some Python code which cut out the conditional text from the original sources using regular expressions to, and saved the results in a different project folder for each build type. I also wrote the code to extract the index and browse sequence information from each HTML source file and use that to create the HTML Help's project files. The content-sensitive help system was implemented using A-links that would be entered as META tags in the sources and which would be interpreted by the compiler automatically.

  The user interface was created using Tix and provided a tree-view of the table of content which showed each of the constituent topic files in the project. I could edit these files by clicking on them from the window and selecting an "Edit" button, which initially called up Homesite but which would later be linked to my favourite HTML editor of the time. I could also view the topic's original source files from Windows Explorer, making it easy to add or remove topics as needed. Compilation was a matter of clicking on the "Compile" button which generated the target files, rebuilt the project files and called the compiler.

It actually worked rather well. The editing environment may have been a bit clunky, and since I'd had no time to create a drag-and-drop table of contents editor, I was forced to enter the browse sequence (or page numbers) by hand, which could occasionally be fairly time consuming, but the payoff was in the compilation speed, which was blazingly fast. The largest help file had about ten thousand pages' worth of information, but it would still compile just in a couple of minutes. Over the years, I extended the system to add more functionality, with the final software providing me with the ability to edit the help using either Expression Web or Dreamweaver.

The flaw with the approach is the same flaw that exists with any in-house software. There's no external support and no information available if you have a problem. That was fine when I was working on my own, as I could fix any bugs as they arised, but as new Technical Authors joined the company, it was impossible to get them trained up on the system, so I ended up doing all the help work myself.

Another issue was at some point Microsoft stopped actively supporting HTML, and had been actively making it harder to use over a network because of potential security concerns. We had to move to a replacement and the market had evolved to the point where such replacements were becoming commercially available. I therefore spent my last few years with the company trying to move the help to one of the standard platforms. When I eventually left, I recommended that the company moved to the latest Robohelp or MadCap Flare.

I don't regret creating Python Help and I still think the project can be regarded as a success. It was used for over ten years and it was run thousands of times to create help files for a large number of releases. It also saved the company a significant amount of money on software licensing. With the benefit of hindsight though, I'd probably have been better to wait for the next release of Robohelp, which did finally support the conditional compilation system that we needed. There was also the fact that the cut down version of the software that led to Python Help's creation was abandoned after a few years anyway.

It was a fun project though, and I'd love to rewrite the thing in C# and add support for other platforms, such as Node.js. Who knows? Maybe I'll do that in a future career.