Open Documentation and "Evil" Governments

ClassicBri

Concerning your most recent show (June 26):

Somehow I doubt that utilizing Microsoft Office as a primary word processor is a conscious choice by governments to be evil.

Being a Mac user, I don't use Microsoft Office very much, but whenever I've encountered a Microsoft .doc file, I've had no trouble opening it through the two word processors that Apple offers (AppleWorks and TextEdit). Outside of that, I have no experience with other word processors. But is there really a problem with certain programs not being able to open .doc? I ask this because if there was a problem, I can see where Rym and Scott were coming from.

Sure, it may not look pristine - but the goal is to freely be able to access the information of governmental documents, not to look pretty.

And yes, open documentation is far better, but I'm wondering if you're just making mountains out of molehills.

One more thing: so if you don't use Microsoft Office, what other word professors are out there? Please enlighten me as personally, I love Word because I like the features it has and how nice I can make a document look in it. I use AppleWorks primarily, but it doesn't have that many options. Is there a good free for-Apple word processor?

Ilmarinen

Yes, OpenOffice is a word processor, and they offer a mac app, and there is a there is a port to OS X called NeoOffice.

Rym

The problem is that, while those programs can open the files, you can't create a fully compatible doc file without MS Office. Many government agencies will only accept electronic papers with specific parameters or formats, and will reject ones created with other programs that are a little off. No editor other than Office itself can create true docs, and any formatting beyond the most basic of text is a heady wildcard.

Furthermore, many closed formats have been all but lost to time due to their closedness. There are very few word processors today that can read the old Microsoft Works text document format. I have dozens of such documents that can be opened in varying states of readability. Because the format of the document as never documented, there is no way to re-write a reader.

Apreche

AbiWord is my WP of choice. http://www.abisource.com/

And Rym brings up another excellent point. The only people who know how closed document formats truly work are the companies that make them. If they do not open up their format and they stop supporting their software, any documents in those formats may be lost forever. Considering that people have huge problems opening documents made with Word97 in the newest Word, this is a problem.

Let's give an extreme example of why this is a problem. Let's say I want to submit a Writ of Certiorari. If the formatting on those is off by even the tiniest bit, the Supreme Court will reject it regardless of the content. If somewhere along the way the document has its format changed, or perhaps a lawyer opens it with a different Word Processor, the formatting will be hosed.

Because everyone knows how open documents work every Word Processor can handle them 100% correctly. Because much of how closed document formats work is secret only the proprietary official software with which they are associated can truly handle them properly.

Rym

In fact, there was an issue in many earlier versions of MS Word where data from other documents and even files on your computer would be stored within the proprietary data of doc files. You could open them in a text editor and see all sorts of garbage or compromising information. Furthermore, you couldn't predict what data would be stuck in there, since the format was proprietary and hidden.

This sort of thing can't happen if the format is well known, well understood, and well documented for all.

ClassicBri

For some reason, older Word documents never even occured to me. Yeaaa, I can see how that would cause a lot of problems...

Apreche

Precisely. If we use MS Word now to make a document and store it on a drive somewhere. Then 10 years from now, Microsoft might not even exist and the way that file format works will still be a secret. It will be very difficult, if not impossible to read that document. You'd have to get a 10 year old comptuer and pirate an old copy of Office. Doing it legally would be even harder and more expensive.

If the document were stored in an open format, then at worst someone could write a small program to convert to a more modern format. If there are no secrets, then anyone can make it work.

There's another aspect to this, and that is knowledge. There is a huge shortage of people who know about mainframes and other old technologies which are still in use. This is because these are secret expensive technologies which few people had a chance to learn about. Things like Linux are free and completely open, so absolutely anyone can learn them for no financial cost. If you use open software you can always find someone who knows it or can learn it with ease. If you used closed software, you have a very small pool of people who know how it works, and they will eventually die out. Many big companies like IBM are having problems as their mainframe employees retire because nobody new knows that stuff. If they had used something open, like Linux, there would be no shortage of employees.

ClassicBri

Then it seems like that's the real problem here. The problem is that people aren't making efforts to switch now, not that people have used closed software in the past. I say this because I can chalk up past closed software usage to "hindsight is 20/20" in that the first 10 or so years of this kind of development, noone knew of the true repercussions of it.

In fact, it seems, in a mindsteam light, that open sourcing is a relatively new phenomenon. And I think that this just comes with the territory - people base new knowledge on old knowledge and nothing really exists in the non-computer world that can be open sourced so no one even thought that it could be done anyother way. God knows that I didn't know what open sourcing was even four years ago and I count myself as more computer savvy than the average person.

Based on that hypothesis, even though it was wrong, people were ignorant to it, so I think we can excuse it. The real problem is not changing based on what we know now and there in lays an even newer problem. Multi-billion dollar companies such as Microsoft don't seem keen on changing their business plan when it's worked fine for ten to fifteen years. Loyalty to old technology comes into play even though that old technology was based on a "shot in the dark" mentality.

I completely agree with you guys though, I'm just laying the blame on a different problem.

Rym

Honestly, people don't need to change. Joe Schmoe can use whatever formats he wants, and I couldn't care less. If he can't send me a file I can read, that's his problem and vice versa.

The problem occurs when a government entity requires the use of a proprietary format in order to access a public service.

Apreche

Openness in computing is not a new phenomenon. Computer and hacker culture began as open source. In 1946 the MIT Tech Model Railroad Club(TMRC) were the first "hackers". In that club were guys like Peter Samson, often credited with saying "information wants to be free". Almost 30 years later Microsoft, Apple and the Altair(the first PC) all appeared in 1975. In 1983 Richard M. Stallman(RMS) started the GNU project and in 1991 Linus Torvalds wrote the first Linux kernel.

So in 1983, as soon as MS and Apple were beginning to build steam, people like crazy old RMS were already moving into action. When they realized that the GNU project wasn't working so well, they pushed Linux throughout the 90's. To say that this is a problem we just figured out is simply false. This was a problem computer scientists realized and dealt with from the very beginning. It was only because big corporations choosing to go against the existing culture in the interest of financial gain that we have these problems. And that brings us back again to capitalism, freedom and bad intellectual property law.

The way I see it, closed source, proprietary software is just a bump the road. Linux kernel version 2.6 and Firefox brought us over the peak, and it's been downhill ever since. Computing has mostly been, and will always be, a necessarily free and open science.

ClassicBri

Scott, I totally understand what you're saying and I agree with you (and also good to learn about the history of this stuff as well, which I didn't know), but old knowledge is based on new knowledge and when you have American culture so saturated in Capitalism, it's not hard to see how it was easy to just bring those same rules to the internet.

I'm not saying it's right, and I like the idea of open source stuff the more I've researched about it within the past year, I'm just justifying for the sake of justifying, I suppose.

Apreche

I'm just trying to point out that this wasn't a mistake that was later realized to be a bad idea. This was something that was a problem which was caused by the deliberate actions of certain people. We are now fighting to undo the intentional acts of others.

ClassicBri

Makes sense - obviously that's your area of expertise and you know more about it than I. Fascinating stuff, though!

GauntletWizard

Bump for mention on the show.

Honestly, that people would think that Open Source and Open standards are a relatively new idea is scary. It's a recipie for another tower of babble: Nobody will be able to talk to each other, and will eventually seperate into distinct and somewhat stagnant cultures.

ClassicBri

It's hardly scary. Just the common number of people don't know or care about it because it doesn't directly affect them. This is hardly unique to computer culture. People will usually choose the more convenient method of doing things unless it noticeably affects them directly.

People buy Windows - Word is there. Convenient to use. Why would it even occur to anyone that it was problematic?

Isn't the scary part of this just that the people who do know it's better are not trying to spread the word and educate the people?

Farragar

I'd like to point out that when you buy windows isn't just 'there'; all you have is notepad and wordpad. you have to pay a good £80 for Microsoft office, with excell (spreadsheets) Power point (presentations) and a bunch of others. Granted, however computers usually come with windows isntalled and office installed, but the manufacturers charge you for office while you 'think it comes with the computer'.

Secondly if you ever used word a sensible ammount of time (something i do NOT reccomend doing) you will see it is a terribly buggy horrible program with all sorts of formatting and alignment errors. (particually when using images, and multiple fonts)

Apreche

Isn't the scary part of this just that the people who do know it's better are not trying to spread the word and educate the people?

Um. I dont' know what you're thinking, but there is plenty of educating going on. The problem is that there aren't a whole lot of people paying attention to that educating.

Colin

There are many reasons government is behind the times when it comes to technology. I spent 15 years working in HR for city government before I quit in frustration. I have many horror stories.

Rym is 100% correct when he states that government agencies won't accept files if they aren't done in the exact format they want them. The HR I worked at only recently started accepting electronic apps and resumes. The format requirements are strict. Even if you are the best candidate on the planet, if your resume and app are not sent in the correct file format, you will not be considered.

When I first started, the city was in love with WANG. They thought it was cutting edge. We used daisy wheel printers. I asked one day whey they don't get a laser printer as they were much, much faster and more efficient. The answer? "What are those?"

I actually got paid 3 hours overtime once because they needed a 75 page labor contract printed out and copied for an early morning meeting. I had to use the daisy wheel printer. All I had to do was line up the page and push print 75 times. I used the time to read a book. Tax dollars paid for the OT.

Apreche

I actually got paid 3 hours overtime once because they needed a 75 page labor contract printed out and copied for an early morning meeting. I had to use the daisy wheel printer. All I had to do was line up the page and push print 75 times. I used the time to read a book. Tax dollars paid for the OT.

Why didn't you just take it somewhere else and use a real copy machine?

Colin

The contract had just been agreed to by labor and management. I had used their notes and type up the agreement in a proper legal format. The city council and the city attorey needed a copy to review the next morning at 8 a.m. so I got 3 hours OT for printing it out on a daisy wheel printer. If I could have copied it instead, I would have.

trogdor9

[...]Tax dollars paid for the OT.

Woah! Tax dollars pay for Operating Thetans now? Holy shit!

Colin

LOL. Government counts on the people not knowing what goes in behind closed doors. :(

TheWhaleShark

Scott: It's not always just as easy as "taking it somewhere and using a real copy machine." The level of anal in government goes deep; it goes really deep. There's often an approved method, and deviating from that method invalidates whatever you were doing.