This document contains only my personal opinions and calls of judgement, and where any comment is made as to the quality of anybody's work, the comment is an opinion, in my judgement.
[file this blog page at: digg del.icio.us Technorati]
joins between two tables, and there are essentially only two algorithms for this: sort tables and match both in a sequential scan, or match every element against every other element. The latter strategy is usually good only if one of the two tables is rather small and fits into low latency memory, as the other will get sequentially scanned and lookups in the smaller one will be usually random. But still the larger one is scanned sequentially. Sorting by key or address and then doing a sequential sweep is also good in RAM, as I have argued previously especially for non sequential writing. Interestingly things may be changing for mass storage, as with
flashbased mass storage conversely latency is about the same or even rather smaller than (single sector) sequential reads (but random writes can be much, much slower than sequential writes), which rather changes the tradeoffs pertaining to database indices and probably much else besides.
Such technology stalwarts as Yahoo, Amazon.com and Research in Motion, the company behind the BlackBerry, have all suffered embarrassing technical problems in the last few months.
About a month ago, a sudden surge of visitors to Mr. Payne's site began asking about the normally impervious Amazon. That site was ultimately down for several hours over two business days, and Amazon, by some estimates, lost more than a million dollars an hour in sales.
Companies like Google want us to store not just e-mail online but also spreadsheets, photo albums, sales data and nearly every other piece of personal and professional information. That data is supposed to be more accessible than information tucked away in the office computer or filing cabinet.
Last holiday season, Yahoo's system for Internet retailers, Yahoo Merchant Solutions, went dark for 14 hours, taking down thousands of e-commerce companies on one of the busiest shopping days of the year. In February, certain Amazon services that power the sites of many Web start-up companies had a day of intermittent failures, knocking many of those companies offline.
The last quote may be true for e-commerce sites, but it is not quite right for cloud computing and application service providers, as there is a fundamentall asymmetry between the two: when sites like Amazon or Google are down it is their business that suffers and their users are merely inconvenienced, as with a shop that is without power, but when an application service provider is unavailable, it is like when a bank closes: the users' data become unaccesible, and their business suffers.
Jesse Robbins, a former Amazon executive who was responsible for keeping Amazon online from 2004 to 2006, says the outcries over failures are understandable.
"When these sites go away, it's a sudden loss. It's like you are standing in the middle of Macy's and the power goes out," he said. "When the thing you depend on to live your daily life suddenly goes away, it's trauma."
network effectswill move to application service providers.
sabiwattsscript. The results are pleasing: doing editing and other light work I can use my Toshiba Satellite U300 (with a 6 cell battery) laptop for 3-4 hours, and 2-3 hours doing hard work like compiling and developing. I have had a look with the useful
powertoputility and while I am typing power consumption is around 12W:
PowerTOP version 1.9 (C) 2007 Intel Corporation Cn Avg residency P-states (frequencies) C0 (cpu running) ( 3.3%) 1500 Mhz 0.0% C1 0.0ms ( 0.0%) 1000 Mhz 100.0% C2 0.0ms ( 0.0%) C3 11.9ms (96.7%) Wakeups-from-idle per second : 81.5 interval: 15.0s Power usage (ACPI estimate): 11.7W (3.9 hours) (long term: 15.0W,/3.0h) Top causes for wakeups: 20.2% ( 19.1) <kernel IPI> : Rescheduling interrupts 18.5% ( 17.5) kicker : schedule_timeout (process_timeout) 17.8% ( 16.8) X : do_setitimer (it_real_fn) 13.9% ( 13.1) <interrupt> : extra timer interruptRunning
find /raises that to 16.5W, and driving up the brightness of the screen adds another 1-2W. The main factors in saving power seem to be:
conservativeCPU governor, which keeps the processor at the lowest speed and power rsting except in the case of it becoming busy for an extended period of time.
transreflective, that it is still readable when directly hit by bright sun light, even if then it takes on a yellowish tint.
sysctl vm/block_dump=1setting, and derived the related settings in the
sabiwattsscript, to increase buffering and cluster disk activity to ensure the disk is woken up rarely. After all laptops are battery-backed, especially when on battery power of course, so caching in memory is fairly safe (even if I have lost a couple of times some files as I forgot to plug in my laptop after a trip and power run out...).
syslog.confto a named pipe as in:
*.=debug -|/tmp/debugto prevent the log events going to disk themselves... As to minimizing disk activity I have found that one should minimize the number of
syslog.conflines that are unbuffered, and the number of running dæmons. This this effect I take advantage of the
run levelsof sysvinit and I use run level 2 is for standalone power saving operation, run level 3 for networked power saving use as a client, and run level 4 for non power saving use with all my server dæmons running. I don't use run level 5, because it traditionally is used to enter GUI mode, and run levels are not really designed to offer combinations of choices (low or normal power, text or GUI), so I enter GUI mode by running
xinitafter logging in in text mode.
More importantly to those of us who are already feeling the squeeze from watching so many of our jobs go overseas to countries with lower labor rates, will business analysts soon be competing with us as well? Will tools advance to the point that many business people will be doing their own application development?
Unfortunately it is not as easy as that: as the cost of programmers in the third world is one or a few dollars per hour, very few business analysts or business people in the first world will waste their expensive time trying to write their own software or executable models.
I think those who wish to remain on the same career path (corporate developer) must continue to learn the business they are supporting, continue to improve their rapport with business stakeholders through communication skills, and continuously retrain themselves as new technology lifecycles appear. This is not much different than what many of us have already been doing.
If you want to remain in the field of programming stay on top of UML, OCL, & MDA.
silver bulletin either the
executable specificationaspects and the idea that analysts and users can develop their own executable specifications or models is very ancient, at least as old as COBOL. It was was described as a simple English like language, as some still say:
so much so that analysts or users could write programs directly in. This did not turn out to be the case, to say the least.
COBOL is a high-level, English-like language which, when used correctly, can resemble a well-structured novel with appendices, cross-reference tables, chapters, footnotes and paragraphs.
O_DIRECTaccess to those files where appropriate (almost always), and they pointed out that on their workstations (top end laptops and desktops) running one instance of the application locally, performance was acceptable. My point was that the cumulative effect of several users hitting the same fileserver would have been perhaps less ignorable.
scratch my itchmindset that is so popular: as one voice in the wilderness says applications still
suckas they are written with utter disregard to common sense frugality of resource consumption (from file metadata accesses to memory). The waste in each is easily masked by the abundance of resources in current workstations (including recent laptops) but what is hard to notice in a single application can become very annoying in the aqgregate, when one wants to take advantage of the boon of modern technology to run more things rather than one thing more slowly (even if not obviously so).
publishingI should have credited one of the classic books on the
dot.comboom, Burn rate by Michael Wolff. Some quotes from reviews to illustrate why:
Although Wolff (Where We Stand) was an early believer in the ability of the Internet to deliver powerful content to a mass audience, by the time he resigned from his own company in 1997, he had come to see the Net as more of a transactional medium.
An entrepreneur's 1994-1996 tale of search for IPO; premise rests on essential tension between East Coast (content) and West Coast (technology).
In the book indeed it is described the difference, even of personal styles, between East Coast publishing (such as Time-Warner) and West Coast (such as Microsoft) computing cultures. Now it is quite ironical that technology, transaction oriented Amazon was founded by a New York investment banker and content, publishing oriented Google as founded by Stanford engineering postgrads. Of course I am talking business model here, because both the approaches are heavily technology based.
The Internet is not, as the East Coast media companies wanted to believe, a new medium in which "content is king." However well crafted the content, he laments, "We could see the 'user' hover for a second, or half second, and alight on a paragraph and then linger for a moment and be gone. Bolt. Vamoose. Vaporize." Mr Wolff explains, "In some perfect irony we had created a new publishing medium," but there was no one to read it.
The more prosaic reality, Mr. Wolff came to believe, was that the Internet was not a new medium, like television or cinema, but a communications technology, like the telephone. It was a glorified phone with wonderful and startling commercial possibilities, true, but still a utility or part of the infrastructure rather than a medium for creative endeavor. The Net is for looking up factual information, such as airline flights or stock prices, or buying things, such as computers or, well, sex.