News:Item-93249
Taking a step back
Sometimes you need to take a step back before you can move forward again.
During 2023 I was working on several fresh initiatives for this website. Most of them centred around the use of IIIF and TEI XML, while another was aimed at empowering editors to curate personalised collections based on their interests. Before embarking on new adventures, however, I felt that two matters demand more immediate attention.
1. Parsoid
In the first place, a breaking change announced by MediaWiki, software on which this website is built, has made it necessary for me to start rethinking and refactoring a substantial part of the way in which CODECS is written and structured (refactoring refers to a way of thoroughly rewriting a codebase without anyone truly noticing it). MediaWiki has been rewriting what it calls its ‘parser’, which refers to a PHP-based system of converting between wikitext and HTML, which is also critical to the way the database gets populated. This new parser, dubbed Parsoid, is intended to replace its predecessor sometime in the future, optimistically within the coming two or three years but possibly sooner. While there is time, that time will have to be spent efficiently given the vast amount of work that goes into coming up with solutions and implementng them at scale.
While in the world of wiki management, I am not the only one facing issues relating to Parsoid, many of the solutions that have been proposed out there assume you have privileged access to the server. On a more restricted hosting plan like ours, however, I’ve had to look for creative alternatives. By the end of last year, I had settled on a number of options, with the help of some MediaWiki extensions I wrote for this purpose, but it is the practical application – ongoing since 2022 – that takes time.
P. S. For fear of losing my audience, I have refrained from being specific on the technical challenges. But for those that are interested, I have begun preparing an outline of solutions, which is available in the new Lab space on this site.
2. Hosting
Over the years, it had become increasingly clear to me that CODECS is outgrowing its hosting environment. It is an extensive and still expanding site – last time I checked, it was in the top 25 of the largest ‘semantic wikis’, i.e. wikis using Semantic MediaWiki, around the world (source: WikiApiary). Size is only one thing though. The greatest problem is presented by external developments: some of the key software components have become more demanding in terms of how they are expected to be installed, upgraded and maintained.
I think it’s fair to say that recently, a critical point was reached. If a maintenance process intended to keep the database healthy goes awry, as happened in January this year, and the only way to become aware of it is by learning your provider has shut down the entire domain, you know a new plan of action is probably a little overdue. Same when you come to realise that there is nothing within your power to terminate the erratic process.
What used to stand in the way of a solution is simply a lack of finances. There is a steep increase in annual costs when you transition from shared hosting services like the Van Hamel Foundation has been using to services that provide greater freedom and allow for tighter control and monitoring – managed hosting, VPS. The Foundation simply would not be able to cover the expenses of the latter. In terms of hosting plans and pricing, there has not been a viable middle ground.
Until recently, that is, thanks to what might be called containerisation. It is a growing trend in shared hosting that can be relatively affordable and gives clients more freedom, even if they cannot go all out on backend options. That’s why we made the move. After a period of testing the waters, I have been in the process of migrating all our websites, including https://kelten.vanhamel.nl, to a new provider offering the benefits of such a package and should be ready by the time you are reading this.
So far so good, but are there any trade-offs? Unfortunately, unless we upgrade, it appears we do not have as much elbow room as we used to when it comes to inode usage, which roughly refers to the number of files one is allowed to store. As a result, there is little to no space left on the server for running a proper staging environment, or doing other forms of testing and development. In practice, it also means subsequent upgrades may come with longer downtimes, much as I will try to keep them to a minimum.
In conclusion
While everything may look awfully quiet on the outside, it has been necessary to go back to the drawing table and reconsider the (then) increasingly untenable situation with our backend. It is heartening to see solutions are finally coming along and progress is being made on both fronts.
It should be stressed that while the costs of hosting are not prohibitive, the fact remains that they have roughly doubled, which does impinge on the small budget of a non-profit foundation like ours. Not to mention the time that goes into all of the refactoring, migration and upkeep, of course.
On that final note, I should not forget to alert our friends, those of you for whom this website is intended and who have motivated me to carry on with the project, to the possibility of donating to the Van Hamel Foundation, if you can spare a little, and helping the website to grow stronger. Your support will be very much appreciated!
