One example of the advantages of interactivity over batch mode are spreadsheets. All the data manipulating you do in a spreatsheet you can program in a scripting language like Perl, so why we use spreadsheets for them? My answer is that spreadsheets are just more interactive - after each manipulation you immediately see the output of it, if you miss you can reverse it or adjust. It's like in shooting: aim, shoot, adjust and repeat. While with programming you need to close the program to a more or less finished form untill you can run it and see the results. Then, of course, you can again adjust the program and repeat the process, but obviously the loop is much longer than in the case of spreadsheets.
Yet there is one advantage of the programming way of solving the data manipulation problem. Once you program it you can run it multiple times with a minimal work overhead. What can we do to have both the interactivity of manual manipulation and repeatibility of programming? All is needed is that the manual manipulation is saved as a script (program) that can be later executed or edited just like any other program (since the beginning created in an text editor). In fact many of us did that for a long time with shells where the history of commands is saved in a file in syntax ready for execution by the same shell. Onother great example of this modus operandi is CPAN://HTTP::Recorder which lets you record your work sessions with a browser as a CPAN://WWW::Mechanize session. Read the article at perl.com at Web Testing with HTTP::Recorder by Linda Julien if you want a good intro to it.
I believe what I've just described amounts to a whole paradigm shift in distinct programming method, and we shell see more and more systems like the mentioned HTTP::Recorder.
Update: I reposted this thought in my wiki at: SavedMacro.
------
We are the carpenters and bricklayers of the Information Age.
Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose
I shouldn't have to say this, but any code, unless otherwise stated, is untested
I don't know.
When I got my first serious job, I observed people logging into machines with rlogin instead of telnet despite (as I knew) the latter's being the newer protocol. "It's faster that way", they said, and at the time I couldn't make any sense why. It took me a while, but I finally figured it out: they were invoking it as rlogin -l theiruser, thus saving themselves from the interactive prompt for a username. (Many of them had also rigged up .rhosts files, so they didn't need a password, either.)
Unix folks tend to prefer batch operations for things they are already familiar with. It's not just repeatability. There's something to say for the "don't bother me" attitude against computers (and not, of course, against humans). Well-designed computer systems *shouldn't* bother us.
It becomes complicated when the same system serves both newcomers and power users. The web, unfortunately, is not a very programmatic environment, despite its success. Modules like HTTP::Recorder and WWW::Mechanize are indeed a great improvement over manual screen-scraping, but they're no match against a good API.
Don't forget that spreadsheets and word processors have been around for over twenty years now. They certainly have their markets, but ask yourself this: how popular are the (recording and) macro facilities in, say, Word? How many people do you know who can record or write a macro to... say, enbolden the first word in each list item? Consider that this is easy to do in Perl — once you know how. Not that I think it's hard to do with macros, but the scantiness of people who *can* do it is telling.
Only if you discount at least a decade of testing methodologies. Recording scripts is the primary way of testing GUIs.
The reason why spreadsheets are more popular than Perl isn't the interactivity - it's the simplicity. This is the reason why I haven't finished my simplified version of Quicken and gotten my family's books off of a spreadsheet. It just takes too damn long to write the infrastructure a spreadsheet gives you. Once you do that, adding new features is a lot easier. But, getting to that point requires more tuits than I have right now.
That said, spreadsheets don't have the same raw power as a well-written infrastructure, in any language. Plus, you can't run your spreadsheet in a batch mode. Remember - 99.999999% of all data manipulations are handled in a batch mode because they need to be. (Credit card transactions, for instance.) That's the entire point of computers - to do the repeatable tasks so that humans don't have to. They are tools - nothing more.
Meditate a little more on the topic, focusing on where your impetus comes from and how that relates to the general tasks of computing. Remember that the largest consumer of computing power for a very long time was the financial industry.
------
We are the carpenters and bricklayers of the Information Age.
Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose
I shouldn't have to say this, but any code, unless otherwise stated, is untested
The reason why spreadsheets are more popular than Perl isn't the interactivity - it's the simplicityI argue that it is the interactivity that makes spreadsheet manipulations simple. With a program you need to understand the whole, with interactive manipulation you need to understand only the atomic actions.
I don't say that the technique I described was not used before - I made an example with the shell history myself, another one I was aware of was Emacs macros, and neniro mentnioned Exel macrocs can be saved as VB as well. I just believe it is very powerfull and is underused, and we will see it applied everywhere and perhaps the words 'paradigm shift' were a bit grandiose, but I think it is new as a general approach to automation.
Excel is just a application, written in some language. That language could even be Perl, for all you know! It is simple to use and simple to extend. It was designed to be simple in the simple case and powerful in the complex case.
Perl is a language. IT IS NOT AN APPLICATION. People use programming languages when the existing applications are either too simple, too heavy, or otherwise unsuited for their needs.
Please meditate on why you are insisting on comparing apples and Porsches.
------
We are the carpenters and bricklayers of the Information Age.
Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose
I shouldn't have to say this, but any code, unless otherwise stated, is untested
You have to realize something - applications are meant to be used interactively. Programming languages are meant to be used without interaction. You're still comparing apples and Porsches.
------
We are the carpenters and bricklayers of the Information Age.
Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose
I shouldn't have to say this, but any code, unless otherwise stated, is untested
You have to realize something - applications are meant to be used interactively. Programming languages are meant to be used without interaction.Once again I compare applications with programming as two ways to solve the same thing, the first one is interactive, the second one is batch mode. So it's all straw man here (and a bit rude in the wording I need to add).
First, this programming technique is indeed a fast way to solve certain kinds of repeated interaction problems, but leads to fragile systems. They tend to be poorly laid out, hard to understand, have lots of things that can go wrong that aren't handled, and so on. You should recognize this description if you've ever dealt with critical business processes that got started as a glorified spreadsheet from a non-programmer. Those drawbacks aren't a big problem for testing suites. But they are if you're maintaining production systems.
Second, my personal reaction to anyone who is excitedly talks about the next big paradigm shift is to label them as someone who likely doesn't know what they are talking about, and almost certainly doesn't understand Kuhn. Put simply, a paradigm worth discarding at the drop of a hat wasn't worth owning, and the paradigm that you discarded the first one for probably isn't either. Paradigm shifts are not good things, they're very expensive messes that sometimes become unavoidable. The value that is perceived in them could not arise but for the far less visible effects of people working within good paradigms before and after the shift.
If the second comment puzzles you, read The Structure of Scientific Revolutions by Thomas Kuhn. Again if necessary.
To the secont I've allready appologised for using a bit grandiose words. Should I delete them? I think not it would make your comment out in void and at least they are not misleading like a wrong code.
Update: Is the meaning of term 'paradigm shift' restricted to what Kuhn related by this term? I am not sure. Look for example at ParadigmShift for a more liberal definition.
Oh, you've hit a pet peeve of mine. I am working at a company that had revenues of roughly $65 million dollars last year. We have grown revenue an average of 72% year on year for the past 7 years. We are looking at a 125% increase in revenue this year, assuming that one third of the projected contracts are actually signed. It could be as high as 200%. This means that 7 years ago, we had revenues of 1.5 million dollars. In 8 years, we have increased 100x. Our IT staff has gone from 2 people to 22 people. Our total staff has gone from 20 to 250.
What does this have to do with anything, you might ask. It's very simple - our systems have grown incrementally with an extremely short feedback loop. And, they are best described as cancers or weeds. The system is a mess and it is more dangerous to modify it than it is to throw out and start over from scratch. Except, refactoring now would cost $500,000 and take a year. Ooops!
Remember - incremental design does not in any way imply successful design. In fact, it almost never does. You end up traversing the twisty maze of:
- Poor requirements (assuming you even got any)
- Inadequate development time (assuming you even got a fixed deadline)
- Zero testing (TIP/DIP, anyone?)
- Inappropriate executive-level decisions
- The wrong people in the wrong positions
- Complete lack of experience in fundamental roles
You are right - a hallmark of most successful modern applications, proprietary software aside, tends to be incremental development. It is a necessary condition, but it most certainly is not a sufficient condition. There are at least three other conditions I can think of that are required that incremental development doesn't even begin to touch.
- Clear and concise development process which includes peer review at every stage and a strong focus on repeatable testing
- Clear and concise requirements
- Experienced management that understands the division between their roles and the roles of their staff
Give me a project with those three conditions and I'll be happy to work top-down. Give me a project whose only bright point is incremental development and I'll quit in less than 3 months.
Another personal story - I worked at an e-commerce firm which had a continuously incremental development cycle.
- We'd release on Sunday night.
- Monday, we'd receive requirements.
- We would have 27 days to complete development on those requirements.
- We released four weeks from the prior release.
Yet, we had no test suites, no design, no peer review ... no nothing. When I asked about design, my director said "Do it on your own time." But, we had incremental development, so we should've been ok, right?
------
We are the carpenters and bricklayers of the Information Age.
Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose
I shouldn't have to say this, but any code, unless otherwise stated, is untested
The key understanding that Worse Is Better is groping for is that as you change which set of normative standards you use to judge something, what is worse according to one reasonable standard is better in another. Richard Gabriel gropes after this idea by comparing two sets of competing norms that are relevant to software development.
So yes, what is worse by one standard can be better according to another. Know the advantages and disadvantages of the approaches you are thinking of, know your goals, and then react accordingly. You stated the advantages of the "record a macro" approach to development. I've let you know the big disadvantages, and gave an example which is very similar to what happens. The disadvantages basically boil down to it being a fast road to having a big ball of mud. I gave a case where those disadvantages are not a problem, and a case where they are. I think that this is useful knowledge for you to have when you decide where to use this (new to you) development practice.
Given that I've stated the disadvantages of the approach, I probably should spare a couple of words on why it works out that way.
Well-organized systems always have some kind of implicit internal theory of operation. There is some sense about what things are to be found where, why they are there, what kinds of internal divisions to look for and maintain. This theory need not be explicitly stated, but it is understood by the people who are proficient with that system. If you've experienced this, then you know what I'm talking about. If you haven't, then Peter Naur's (who is the "N" in "BNF") excellent essay Programming as Theory Building might help. Unfortunately I don't know of copies on the web, I read it in Agile Software Development.
Note that this organization does not have to be there from the start. There are many ways to emerge an organizational structure iteratively - XP people are big on trying to do that.
However this kind of organization needs a lot of contextual information to find its way into the development process. Similar things go together. Information about why things are done in a particular way circulate among the developers. People spend time thinking about how things should be done.
Opportunities to include that kind of contextual information are sorely missing from the "record a macro" process. You do X then Y. No information is captured about what the person was thinking when they did X. If you think you see something wrong, X should really have been X', there is no context to use to figure that out. The code you get is not factored in any way, shape or form. While the person who recorded the macro may have had plenty of knowledge about what to do with various exceptional conditions, none of that knowledge is captured in the code.
The result is exactly as I said.
As for my comments on people talking about paradigm shifts, I meant that to be useful advice as well. It was criticism, sure, but it was meant as constructive criticism. Let me explain how.
First of all I was doing you the kindness of letting you know how I (and presumably other knowledgable people) are likely to react when you run around waving the words "paradigm shift". Which hopefully will make you a little more careful of those words in the future.
Furthermore I tried to explain why I react that way. That explanation attempted to summarize the most widely missed point in a classic work on the progress of our understanding. If you grasp that point, then you'll understand something key about learning. Perhaps you'll even come to share my skeptical response to people who begin extolling the benefits of joining in the latest "paradigm shift".
I apologize that my lapsing into a form of terse information presentation apparently resulted in your misunderstanding my intentions. All that I can do is clarify what my intentions were and hope that they make sense to you in retrospect.
UPDATE: By schlerotic I meant sclerotic. Fixed.
By the way count the 'paradigm shift' for my lazyness to come with a better word.
Correction. You actually can run a spreadsheet in batch mode by driving it using ActiveX. You can do this from Perl using Win32::OLE. There are limitations to the approach, but it can be a useful approach.
------
We are the carpenters and bricklayers of the Information Age.
Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose
I shouldn't have to say this, but any code, unless otherwise stated, is untested
However I'd also maintain that Access, and applications used in Access, are also not meant to be batched. As a sample indication of why, consider the tendancy for issues to cause a pop-up to appear which practically demands human intervention. (You can get around that with Win32::GUI, but layering hacks on hacks like that is again fragile.)