========================== Textbooks and other news ========================== Another fairly usual month passed by here: things move in the same directions as they used to, whether good or bad. Routers ======= This month's new computer issue: the old router (TP-Link TL-WR841N/ND v8, with LibreCMC) occasionally shuts down its Ethernet and Wi-Fi interfaces for an unknown reason, so I have to reset it then. It happened three times so far, always in the early morning. Fortunately I had a new router stashed already (TP-Link Archer C7 v5), so unboxed it and installed OpenWRT on it, though did not switch to it yet: just ensured that it is ready to be used. Backups ======= The new router was sitting boxed near an external HDD, which I decided to also unbox and turn into an additional backup HDD while at it. Recently I thought that it would be nice to add single-device redundancy to backups, perhaps some error correction codes, so that partial damage to them (bit rot, as opposed to a full device failure) would be repairable using just a single device: like par2, but faster and aiming updates, or like RAID 1/5/6 on partitions of the same device. ZFS has the "copies" parameter to set the number of copies for each block, while Btrfs has the DUP profile (which is used for metadata by default, but not for data, and which is like ZFS's "copies=2"). I decided to use Btrfs on top of LUKS this time, since I already have ZFS (as well as LUKS with ext4) backups, and it is potentially useful to employ varied technologies for those. Also have set Btrfs to use sha256 checksums instead of crc32c this time, decreasing the chances of collisions. Looks like a nice setup for backups, even though I had issues with Btrfs on work servers in the past. Additionally, I created checksum list files with sha256sum(1) and find(1), to verify the backups independently of the underlying storage and without comparing to other backups (rsync --checksum), and updated the personal data storage note accordingly. Textbooks ========= Speaking of backups, as well as of libre textbooks mentioned in the previous post, I rediscovered openstax.org, which has good textbooks licensed under CC BY, and added those to my public data backups. As well as some books from libretexts.org: some of its subdomains geo-block requests from me, but commons.libretexts.org is available. Many links lead to 404 errors, and some of the books are messy (not poorly written, but apparently the builds are semi-broken), but there are seemingly fine ones. Skimmed Wikibooks again, too, but those are rather disappointing. Noticed that those textbooks, and documents in PDF format found online in general, tend to use the US letter paper size, which is something I did not pay attention to in the past, but now that I am considering printing them, and since A4 (along with the metric system) is prevalent here, will have to deal with that. Then found that the pdfarranger program appears to be suitable to cut margins on alternating sides, so that they could be prepared nicely for binding. Though it takes values in percents, rather than millimeters, and while it would work for cutting the sides to fit width, I did not check yet if it will work to fit the height precisely. But even if that will fail, I could still use the libraries it uses, or adjust it. Would be easier if sources of those textbooks were available: the licenses allow to manipulate the books, but do not make it easy. Actually CNXML (with MathML) sources of some OpenStax books are available from their GitHub repositories at , and versions of some textbooks converted into Markdown with MathML are at . Those should be more suitable for different paper sizes than PDFs are, but once I tried to convert the CNXML sources into HTML with cnx-transforms, found that it does not label or enumerate figures and equations, which makes it unsuitable for printing. The cnxml repository contains cnxml_render.xsl, which tries to label and enumerate those after some tinkering (removal of remote and unavailable dependencies, resulting HTML and MathML adjustments), but still unreliably (or maybe I used it wrong). There are other repositories that seem potentially relevant to document compilation, but with larger projects, even more dependencies, and equally little documentation. There is the cnxml-help repository, which is supposed to specify CNXML, but it only contains two strange placeholder images: of frogs, according to a commit message, yet I do not recognize those as frogs. Which reflects the state of its overall documentation. For the potential printing of textbooks and other backups, I decided to check a large local office supplies company's website (komus.ru) for paper, and possibly for storage shelves. It welcomed me with HTTP error 503 (service unavailable), while to curl, links2, and eww it served status 401 (unauthorized), and when I followed a link from Wikipedia, which probably added HTTP "referer", it did a few redirects and served a page saying that it blocked me as a robot, suggested to contact them by email to fix that. I wrote by email, describing the situation and attaching a HAR archive from Firefox, as well as curl -v output, and on the next day they asked me for a screenshot of the error, an IP address, and an organization's taxpayer number. This seems to match the unhelpfulness of ISP tech support, approaching that of government services, which seems a little strange for a store, at least on the stage when you are trying to buy goods from them. Anyway, somehow it worked later, while I gave up on that conversation, and later learned that the website is additionally not available from foreign IP addresses, serving a page suggesting to "turn VPN off". Somewhat related, I came across a public data hoarding thread on HN, . including a link to , which is blocked here already, but serves a lot of books and assorted but amusing text files, including small and humorous articles. Now I wonder about more compact and portable textbooks: printing-friendly and with compact files (for an easier digital storage), both of which can be achieved by providing source files, and preferably easily and reliably buildable ones, likely LaTeX, and being careful with illustrations (simple and monochrome-friendly ones, probably SVG). Can also consider abridging the contents: take something like the Wikibooks physics study guide, with only core formulae and definitions (or, in case of mathematical theories, with axioms, main definitions and theorems, sketches of trickier proofs), encouraging the reader to fill in the gaps, draw (some of) the illustrations (akin to coloring books, "connect the dots"), to compose (some of) the problems/exercises themselves, which should be a more creative and potentially fun process than simply following a textbook and solving the provided problems. Though creation of exercises, even with a guide and usage of random numbers (similarly to solo RPGs), is likely to have the downside of the reader already knowing what went into the problem construction, which formulae to make use of to solve those. Maybe that practice should be encouraged, but with some problems still provided, resulting in something between a study guide's 25 pages and an actual textbook's 1600. Though STEM books are fairly information-dense as it is, and those on humanities would be harder to compress without omitting information, so perhaps it is more useful to focus on having sources with compact illustrations (either finding or composing such books), rather than on experimental adjustments of contents. Besides, one could mark portions that can be omitted, such as additional exercises and explanations, allowing to compile different (tiered) textbook versions out of a single source. Or, as commonly done already, provide an additional problem book. I searched more for such libre and open source textbooks on GitHub, and found "Introductory Physics: Building Models to Describe Our World", , which covers some of the common topics (though I did not spot optics, thermodynamics, or quantum mechanics there), in a way similar to other textbooks. It needed a couple of fixes (one of which was awkward though: siunitx aims numbers without constants or variables for quantities, so one has to call pi a digit or disable number parsing altogether in order to represent a number like 4\pi e-7), but then I built it fine: with its US letter size first, then tried A4, and finally A5. There are some warts with A5, mostly on long equations (which can be split) or multicolumn tables, and some images seem to be scaled down, though all those are fixable. TeX sources for textbooks on STEM subjects are easier to find, on humanities are harder, though maybe in part it is so because the latter group rarely requires complex typesetting. Typesetting =========== Once again I tried to find a TeX specification to properly learn how it works, and possibly to employ some of that for the VMS markup as mentioned in the previous post, but apparently there is no freely available specification (that is neither a command reference nor a tutorial), in which it is similar to ISO standards. The approximate process (macro expansion, followed by primitive control sequences, combined with use of external programs, producing "boxes and glue") is mentioned here and there, but it is quite far from a comprehensive specification. So I decided to look for the specification of a relatively new typesetting system, typst, and have not found much, either: similarly to TeX, there are tutorials and a user reference, but no specification in sight; the closest thing I found is docs/dev/architecture.md in its repository, which outlines the process, but does not specify the system. Though its website's domain name is typst.app, which hints at this not necessarily aiming to center around a specified language, but rather around a single program. And it sells some services, which may explain such an organization. As a side note, I failed to install typst via cargo, which complained about typst-kit not providing features needed by typst-cli; unsure if it is just a broken package, or the way Rust infrastructure is, since there are issues each time I try to use it. Then I looked into SILE: it ships a grammar (though its official one is still that from the reference implementation), and actually looks like a fine system, which has both XML and TeX-style syntaxes. It is written in Lua, and provides Lua scripting, which is something I thought would be nice for a typesetting language: to reuse an existing one, rather than to invent its own awkward language for macros, though then it has macros as well. As other typesetting systems, it provides a dedicated syntax for mathematical formulae, but apparently it is translated into MathML, which can be used directly as well (at the cost of being very verbose). Chapter 9 of The SILE Book, "The Nitty Gritty", describes how it works and the models used, which is the kind of specification I was looking for with the others. While I have not tried SILE, it looks nice overall after skimming the documentation. MathML itself is thoroughly documented. It is a declarative markup language, not a typesetting system, but could still be a starting point, and maybe later I will read on its rendering techniques. Though rendering may be relatively straightforward once all the elements are clearly defined: traverse the parsed MathML tree, put together boxes and glue, then render those (while going through the regular pain of text rendering). Then I tried and failed to find a troff specification, beyond tutorials and user references, but played with its eqn(1) a little. There are program sources which one may use as documentation, but those are not suited for a quick skimming and comparison. At least DVI has an unofficial description, and PostScript's reference looks fairly comprehensive (it is not just a command reference, but also a language reference, complete with an introduction and explanations). For document preparation, perhaps the most practical way is to keep using LaTeX while treating the lower layers as opaque, or possibly XML with MathML, akin to CNXML (but with working tools and documentation other than supposedly-frog pictures), or MathML directly in HTML (though HTML LS is not exactly a fixed specification now), but typesetting may still be interesting to play with someday. Maybe I could update (or rewrite) wwwlite, adding MathML support there, and switching away from GTK, decoupling rendering from the interactive interface. While for study notes (primarily problem solutions), I consider adjusting my usage of mostly plain text with Emacs Lisp embedded for basic calculations and some Unicode for compact representations of basic formulae, and occasional embedded LaTeX for more complex formulae, to Lisp covering all those cases, convertible into LaTeX (e.g., as with SymPy, but Python's non-lisp syntax is not as clearly embeddable into plain text). A Lisp-style syntax could work well for formulae instead of LaTeX (let alone MathML), likely focusing more on semantics (which makes more sense for reading as plain text, though presentation-oriented languages may be simpler and more flexible). I tend to think of URL references as of tricky parts of markup languages usable as a plain text, but compared to formulae, they are relatively straightforward. Food ==== I tried a new syrniki (small cheesecakes) recipe, which I liked: 280 g of low-fat tvorog (quark cheese), one egg yolk (though approximating that with whole eggs is fine), 40 g of flour, up to 40 g of sugar, some vanilla. Combine those, roll into a cylinder, cut into six smaller cylinders, round them a little, place on a baking tray lined with parchment paper, bake for 10 minutes at 180 degrees Celsius (or fry on a pan a little, and then bake for 5 minutes). Quick, fairly easy, protein-rich, and tasty. I also looked into long-term food storage a little: usually I am not into survivalism or emergency preparedness, but the way things around here seem to be headed, including the economy, suggests that it may be useful to convert some of the quickly inflating currency into a food stockpile. Apparently finding proper Mylar bags is going to be tricky here, though there are some stores with those (selling them as accessories for freeze drying), and could try ordering those from China, if I will figure how to access AliExpress without it freezing Firefox. Tried to search for articles about local long-term food storage practices, but only found SEO spam, translated or generated texts. Coincidentally, a new Technology Connections video about freeze dryers came out soon after my little investigation, and as other videos on that channel, it was nice and interesting. Email ===== Occasionally I wonder why there are no lightweight webmail projects around, and considered writing one. At first I thought to use Maildir and sendmail, then it seemed like IMAP and SMTP would be better to use, allowing to not run it as a superuser or per-user processes, but then noticed that since I usually set IMAP and SMTP to require TLS, that would be an unnecessary dependency for a lightweight MUA. Now I am more inclined towards Maildir again. I tried to initialize a Rust project for that, to poke at it again, with a dependency on the "imap" crate, but yet again Cargo took a few minutes to update the package index, then failed because of my "old" compiler version, suggesting to do a package manager's job and pick the last dependency version that supports this compiler version. I tried to do that, but then there were other packages not happy with that version, and I gave up: it is just too bad. It is particularly odd for a language that is about performance and safety, to have a package manager this slow and aiming to ignore the standard infrastructure, while asking the user to do a good chunk of a package manager's job manually. Which is a shame, since the language still looks promising, despite this infrastructure and its awkward syntax. Noticed that when I see projects written in Rust, I expect this broken build process with a suggestion to "curl | sh" things, and probably covered in emojis, rather than something comparable to C, but safer. Though I rarely need a webmail, and rather short on time, so ended up not starting that project at all. In other email news, I learned about Dovecot's LAYOUT=fs option for Maildir, as set in mail_location: . It disables the Maildir++ directory layout, making Maildir compatible with the way MUAs (including mu4e) tend to handle that, so running MUAs on the server and on a personal computer does not lead to directories duplicated with slightly different names. Renamed existing directories manually to match that setting, though an automated conversion would be tricky, it seems: Dovecot itself refuses to convert mailboxes between formats with different folder separators, and I have not found such converters among other Maildir-related tools. Looks like it should be an easy task, though the Maildir++ specification is vague enough to make it quite error-prone; possibly this is why there are no tools for that around. And I had to migrate one of my mailboxes to a new server. Tried to simply synchronize it using mbsync, but that led to duplicated messages (as I noticed when decided to check it with mutt; mu4e did not show the duplicates). Apparently this happens with mbsync after it adds differing X-TUID headers, and then one may have to remove those, and deduplicate the files. Fortunately I had a Maildir backup in a tar archive, so simply wiped the Maildir, restored from the backup, uploaded to the server with rsync, downloaded back with mbsync. Then it was fine. Work ==== There are still urgent tasks at work. For once implementing an open protocol, called TLS (not Transport Layer Security, but Technische Lieferbedingungen für Streckenstationen). The initially provided documentation in Russian (with some highly bureaucratic notes around it, chunks of which I saw) and English (TLS 2002) did not describe the packets that were sent by a test remote device, but I found a TLS 2012 documentation in German (which I had to read with a dictionary or a translator), which did (but I still had to check what and how is implemented on the used controllers). The protocol aims to be general, which comes with drawbacks, but it is sensible overall, compared to the many proprietary ones used for similar purposes. It aims always-on connections for both actuation (to keep signs and messages shown in the manual mode) and data collection (to collect "spontaneously" sent data messages), which is a little awkward to implement in Haskell: while for data collection it fits the current model without adjustments (accepting tasks and spawning independent per-task threads, connecting to a node, streaming the results from its multiple traffic detectors, one per lane, indefinitely), the actuation process has to keep a single connection to a node that may contain multiple VMS, while new independent tasks for those arrive occasionally, and actuation status responses should be returned for those, so the program should read packets from multiple connections and new tasks simultaneously (in order to respond quickly), which is a job for either an event loop or multiple threads with synchronization. Inter-thread synchronization can be quite error-prone and verbose, yet it is more or less idiomatic for Haskell. For now I made an event loop with Control.Concurrent.Async's waitAny, to avoid explicit threads with synchronization between them, but it would have been more straightforward if the I/O functions were asynchronous in the first place. I generally like Haskell, but not the blocking I/O, the exceptions, and a few other things. Among other tasks was debugging of the project mentioned in the "useless code" post from 2018 here, which is not actively developed. Now it turned out that some frontend changes related to hurriedly introduced new functionality half a year ago broke some of the old functionality, and nobody noticed until now. The frontend development is outsourced, so I added a workaround in the backend, which uncovered further frontend issues, which cannot be worked around, like a seemingly hardcoded user ID in settings (also a sign of hurried work and shipment). I would feel bad for its users, but apparently nobody really uses it, except for occasional inspections or presentations. Other news ========== - Had my sleep schedule disrupted by a combination of urgent work and investigation of things like food storage, printing, and typesetting systems, as well as some physics problems. Had unplanned naps as a consequence. - Yet another law was passed here, voted for unanimously as usual, punishing for expression of more views. I would have violated quite a few of those (which are applied retroactively) already if I was more chatty on less technical topics, not to mention that those lead to more and more Internet resources blocked. - Checked vacancies abroad yet again, confirmed again that those are dominated by Java. Some Python ones are available, mostly for "data engineering" and "data science", often additionally requiring technologies I am not familiar with, and often at startups, which do not seem to provide a good job security. Articles and threads like suggest that much of commercial programming is like that these days. - Discovered , a nice game. Apparently based on nand2tetris courses, which are hosted on Coursera, and this is how I found that Coursera is now among the many blocked resources. Anyway, the game is nice: the subject is interesting, and it guides you, but you still basically build a simple relay computer there, and then program it in the "software" part. I had it long planned to go through some CPU design course, likely using VHDL (or maybe Verilog), to learn those at once, but postponing it, since there is plenty of other stuff to learn and to try. But at least touched this subject with that game, which was enjoyable. - Working through the physics textbook at about the same rate as before, a chapter per month; finished chapter eight recently, reached chapter nine problems (on rotation of rigid bodies). Other studies and exercises also go slowly, but persistently. Except for ear training: not practicing that much, but maybe will resume in the future. ---- :Date: 2024-11-14