Sysop
I’ve just put another round of optimizations into the system. In this case caching the p-code generated by php into shared memory This should usually result in faster page display where content is changing rapidly. Posts with fast growing comments should get the major benefit from this. This should allow the site to run on its existing hardware for quite a while without the trauma of hardware upgrades.
This should be the last of the back-end optimization updates. The next round of updates will concentrate more on usability.
Sysop
Some comments on Sunday about this site performing well induced me to have a look at the traffic over the last week. I was extremely surprised at how much load the system can now take without stalling. From the Wordpress blog stats, It turns out that the traffic for the week was 34.6% higher in page views compared to the week before. It was a staggering 61.9% higher than it was two weeks before. It makes me more confident about running the site at a technical level towards the 2011 (or earlier) election.
I’d love to take credit for this (and I will in part). However the reason people read the site is largely because of the content, both from our posters and our knowledgeable and informed commentators*. This is a completely voluntary site run on a very loose cooperative arrangement - it just shows what kind of talent is there to be tapped from the grass-roots. A special mention for Julie Fairey from The Hand Mirror for her election evening coverage who kept everyone coming back for more. Despite being offline for comments for 19 hours, it is still in our top ten days for traffic.
However that is just the visible portion of the site. We’re reliant on a number of technical supports.
Webfarm also deserves a strong mention. Their systems have been utterly reliable since I moved the site there in Feburary. Currently the site runs on virtual private server with 512MB of RAM running Fedora Linux with unlimited local traffic costing $159+GST per month.
However, my biggest kudo’s has to go to the open source people at Wordpress, Fedora project, K2 theme and the wordpress plugin authors without whom this system would have failed dramatically under the load. The site now runs without any code kludges apart from some css styles for the display. It is quite a different site from what was running last year and failing under load.
* Well almost all commentators - I specifically exclude the annoying, largely incoherent, and definitely anti-social trolls. To a man, they seem to mainly be act supporters** hyped on their badly self-assessed intelligence, in love with their own egos, and who take the credo of individualism to the point of being incapable of meaningful social interaction with others. It is the one area that I pity John Key in - he is going to have to deal with adult versions of these clowns. He also doesn’t have the “send to spam” button. However he has done well with an early ban.
** But not all act supporters are trolls
Sysop
There will be some scheduled maintenance at the hosting company tonight between 10pm and 6am. During that period the site may be offline for some short periods.
So if you cannot get the site for a period, then just try again a little time later.
Yet another record breaking day at The Standard yesterday. We keep breaking our own records for numbers of page views and numbers of visitors. The site does seem to be be running better during the peaks since the database tweaking.
I’ve brought the site up to date. All the various hacks that I put in this years have now been removed as the software and plugins have now been upgraded.
The main area of difference is in the re-edit, which is no longer in-line. I’m still tweaking that in places for look and feel. I’ve very interested if this gets rid of the problems that people have been having with the older version recently. There was one bug in it that I sent the bug and fix to the author.
Let me know of issues you’d like addressed in the current look and feel, and any bugs.Currently the only known bug is the missing caption graphic (working on it).
I’ll be doing some more of the wish-list stuff later, but it is unlikely to get into place before the election. I have tweaked the database parameters to give better performance when the system goes under HEAVY load, as happened twice last week.
There will be a server upgrade on the 16th of October about midday. I’ll remind people closer to the date.
Lynn
After a lot of work both by a few e-mails from people here and by the good services of some people on nz.comp (google seems to be a little behind on the messages) I finally found the link to the malware site that was attached itself to the site footer.
The material that it was trying to introduce to people reading the site may include various forms of backdoors. It would be adviseable to run a good virus scan on your system if you have read the site in the last couple of weeks. Corporate systems shouldn’t have had an issue because the site it was linking to has been a well known chinese malware site for a long time.
The anti-virus/malware scans missed it at the server because it was a new variant of an old problem (the same one I had in march), targeted specifically at wordpress sites using what is evidently is still a open vunerability. My own checking of the site missed it because it had managed to leave all of the file attributes of the file (size, times, etc) exactly the same as the origionals. My attempts to see what people were reporting had failed because it only emitted the malware link out periodically. A dump of the web page at the client side by Stephen Worthington allowed me to see exactly what it was doing.
The vunerability it was exploiting was meant to have been fixed in wordpress 2.5, however they seem to have found another vunerability. The downside of having open source software is that it is possible to read the code looking for holes. I’ve done some things to reduce possible problems, but I now have MD5 hash check of the files running periodically which will fix the problem if it happens again. I’ve also reported the details to wordpress and a couple of other sites.
But there are some very creative people out there writing this stuff, and evidently this site is popular with them.
Lynn
I’m on ‘holiday’ for the next few weeks to contribute some campaign work.
However it will also give me some time to do the next set of upgrades to the site sometime during that period. I will be putting in the acculmulated upgrades for existing packages and testing them.
Here is your opportunity to say what kinds of features you’d like added. I won’t promise to put them in, but I’ll at least have a look at them. Remember they have to stay consistent with the look and feel of the existing site. Have a look around the Wordpress plugin area.
Lynn
One of the links to The Standard this week was from Reading the Maps on the topic of Playing the history card. The very long post is a interesting commentary of the differences between the various wars of the 19th century that made up New Zealand formation and well worth a read.
The author rightly castigates all and sundry (including posts on The Standard) about their response to Key’s history card with confusing the Musket Wars and the Land Wars. In our defense, I’d point out that it was rather confusing about what Key was talking about. Obviously his Australian advice was probably of little help.
A lot of old news is available from Papers Past from the National Library. I dug up this interesting gem from the New Zealand Gazette and Wellington Spectator, August 21st 1839. HISTORICAL SKETCH OF THE COLONIZATION OF NEW ZEALAND. Perhaps John Key could read this rather long view from the settlers at the time. Amongst other things it is complaining about the ruffian element, mainly Australian, disturbing the peace and why the British government should do something about it. The Australian aspect of the complaint sounds vaguely familiar.
It is one of the curious ironies of the local net. Most current or recent news is available via the web, and natlib has a lot of the old material available. I wanted to find out the news at the time about the Auckland Harbor Bridge being built (try this Bridging the Gap). However newspapers from that era appear to be inaccessible on the net. There is a dead area from about 1920 to about 2000.
Could someone give natlib some more resources to bring more of this online at Papers Past? Just at present, virtually every newspaper referred to as an Auckland paper at the Auckland City Library is not on Papers Past. How about putting The Standard version 1.0 online as well - that’d be interesting.
Natlib needs to make the whole sluggish site a bit faster!
On the odd occasion I have time to read outside of the confines of The Standard and its ever increasing brawl of entertaining comments. I noticed we don’t have a external reading list, and it is within the range of my writing skills, so here are my oddities for the slow weekends….
From quote of the day on the Linux test box. It shows a correct appreciation of the art of development.
Scott’s second Law:
When an error has been detected and corrected, it will be found
to have been wrong in the first place.
Corollary:
After the correction has been found in error, it will be
impossible to fit the original quantity back into the equation.
Toms Hardware (great site) has a article 10 Ways to Beat the New Hands-Free Laws for the toy freaks on hands-free cellphones for cars. I loved the description of users…
..in recent years, headsets have acquired a nasty stigma. Depending on your point of view, Bluetooth headset wearers might either look like cyborgs, telemarketers or simply jerks.
If you already wear one, don’t take this personally. It’s just that some people have been holding out for as long as possible to avoid looking like unstable people that talk to themselves.
Not only do I get frustrated with microsoft software to the point that I don’t use it - it turns out that Bill Gates has the same kinds of problems. Full text: An epic Bill Gates e-mail rant. Very entertaining and I’d hate to moderate the comment stream on that site.
On a more serious note I dug around my favorite site at the Economist to find this gem. Down and dirty on an alternate approach to geothermal energy. We live on top of a slow nuclear reactor called the Earth where heat is generated from the slow breakdown of uranium and other heavy isotopes. Why would you bother with all of the problems with fast nuclear reactions when you can push down pipes in a variation of the mohole project to tap geothermal energy. It is a well known under-utilized technology set that doesn’t have a lot of gotcha’s.
Continue reading ‘Geek’s view’
Yesterday’s site issues were due to a combination of inadequate Expires (causing a lot more files being transmitted) and the web-crawlers grabbing the updated site.
It blew out the memory. I’ve limited the simultaneous sessions to prevent that until I get more RAM. Bandwidth usage for this month is indicating this would be a wise precaution to speed page delivery, even without the site updates.
I’ve done a few tweaks to the front page of the site to speed finding the current state of play and balance out the blogroll. Because of the volume of posts and comments, I’ve increased the size of the latest of both. This means you will be able to see who is talking across posts more easily.
The access to the older posts are now inside ‘Archives’ at the top menu.
There are a number of features that I’ll be turning on over the next week or so as things settle down. But none of those should structurally change the site. The back-end changes are quite effective.
Let me know of the gripes and suggestions either as comments in this post or e-mail (but avoid the political sniping). I will do what is feasible, while keeping a close eye on that bandwidth and memory. Hopefully this is the last of the upgrade issues apart from the disappearing IE comments box that I still have to fix.
Lynn
