We have different design goals and constraints when we’re doing one [wizarding] or the other [engineering].
With engineering, we’re tasked with the long term maintenance of a piece of software, and we also
become concerned with many lower-priority properties, such as its performance.
With wizarding, we’re often best served by designing for ease of throwing code away rather than
maintaining it. Well-engineered code is irreplaceable, but well-cast spells should be transient.
In this post, Ted Kaminski talks about two different styles of programming:
wizarding and engineering. At first, all startup prototypes/Minimum Viable Products
are created as fast as possible: we need to see if our startup idea is viable first.
If the business grows, the developers need to transform this prototype to a stable
product, able to grow and include new features.
Developers need to shift from the wizarding attitude (everything is temporary)
to an engineering mindset (long term maintenance, the software needs to work for a long time and is used
by many users).
These two "styles" -and the transition between them- reminds me of
the Makers vs Menders difference. A Maker is a developer who love to create
stuff quickly and are excited by deadlines and time pressures.
On the other hand, a Mender likes older codebases, is satisfied by fixing longstanding bugs
and making the whole system more stable and more secure.
How do we do switch from one style to the other, though?
The author suggests we should write enough tests to be sure everything is working,
separate big applications into microservices when it makes sense,
and rewrite critical parts of your system in languages that can offer static (or gradual) typing
and good tooling.
Separating concerns into their own services -when it makes sense and does not lead to
a distributed monolith- can help reason about the system as a whole, and making
each service faster and more stable, especially when paired with the right dose of testing.
I'm also a fan of static languages such as Rust (even in the browser!), that can
feature good tools such as cargo, and an expressive language that helps developers
implement critical systems with fewer errors.
I'd also like to suggest to recognize when the project should move from wizarding to engineering.
Failure to understand when the project needs to transition to the engineering state, and plan for it,
is probably the most unappreciated problem I encountered in my career so far.
If the team can't slow down and fix growing pains, they will only get stronger and mutate into
outages, and the team won't stop firefighting production issues.
My suggestion is to keep note of small issues, or tasks that are taking longer than expected
for technical reasons, and expose them during a retrospective. Having senior engineers
in the team also helps, especially if they are experienced with the project's technology stack.
So, I hope you liked the first post of Interesting Links, a new column
where I highlight interesting articles, with my own comments and thoughts.
If you liked this post, you should totally share it on Twitter, or support me on Ko-fi.
Some time ago, I wrote a compiler for a fictional assembly-like language.
As this compiler (named hrm-compiler) performs some optimizations on the code, I wanted to
know if the optimizations were breaking the compiled code.
To test them, I wrote a small interpreter (hrm-interpreter) to run
a JSON version of the compiled code, and then embedded the webassembly version
in the bare-bone debugger I wrote as a web application.
If you want to try, the best way to get started is the Rust and Webassembly Book ,
a tutorial that lets you write a Rust crate that can interact with Javascript and publish it to npm.
I followed the tutorial, and managed to create a wasm wrapper of hrm-interpreter,
named (such fantasy! amaze!) hrm-interpreter-wasm. Let me explain what I learnt while creating it.
Update: the blog post you're reading right now was written in late 2018: some suggestions may be outdated.
Let me suggest a newer article about Rust and WebAssembly, written by TopTal's Peter Suggate:
WebAssembly/Rust Tutorial: Pitch-perfect Audio Processing.
Write a different crate to support wasm
If you need to convert Javascript types to Rust ones, I'd suggest to write a specific crate.
It's better to think about it as if you're writing a wrapper. This new crate
lets you write wasm-specific code, without modifying your library to support
this new platform.
Not every library needs to create an ad-hoc crate, though: handlebars-rust offers users a flag
to turn off filesystem-based I/O. This may work if your main library does not expose complex types,
or you are already handling platform-specific stuff.
Don't force file I/O in the main library
What does an interpreter do?
read a file
convert its content to a list of operations
execute each operation until the program ends or some error is raised
writes the output of the program to a file (or a terminal)
Unfortunately, you cannot read or write files in a wasm environment - you cannot perform I/O!
So we need to find another way: let's look at the list again.
read a file and extracts its text as a huge string
convert the content of the string to a list of operations
execute each operation until the program ends or some error is raised
captures the output of the program into a string
writes the output string to a file (or a terminal)
So, we may notice that the focus is now on strings, not files.
As javascript can perform I/O (read files via FileReader, generate new files
via FileWriter, or even do HTTP requests!), our wrapper crate may just
take a string, pass it to the main crate and return the output string.
In my case, I had to create some new methods to sidestep I/O from files.
There are other cases where you may need to modify your crate
to port it to wasm: I suggest reading Which Crates Work with Wasm?
to learn more.
try to expose as little state as possible - use functions instead
Javascript cannot manipulate complex stuff, such as options, so you cannot export whatever you want.
Before wasting several hours trying to understand why you cannot "export"
your beautiful struct with your awesome Option<Result<Beatiful<Type>>>,
check out wasm-bindgen's Supported Types documentation chapter. Trust me :).
To work around it, in hrm-interpreter-wasm...
the main structure has no public fields
the internal state is serialized to a JSON-formatted string
data from Javascript are also JSON-formatted strings
This is not the best interface, but it works for now.
"Now" means that I'll fix it once I integrate some work-in-progress stuff, such as the new Rust-based compiler and sourcemap support...
If you can, try to avoid serialization, as it's pretty expensive... and we're using Rust in the
browser to run code faster! ;)
Browser-side: use webpack 4!
Please note that this article was written in late 2018 - theTopTal articleshows a different way to load wasm files by usingwebassembly.compile.
If you want to test your new crate in a real application, you may want to use webpack.
Starting from version 4, Webpack ships an internal wasm loader: it makes integrating wasm-based libraries
a breeze!
react-create-app offers a very simple way to get started. If you have an older project,
I recommend you to switch to react-scripts ^2.1.1 and webpack ^4.26.
I am not sure you need to do this in new projects, but I had to "eject" the webpack configuration (webpack.config.dev.js)
and exclude ".wasm" files from the extensions managed by the "file" loader.
// "file" loader makes sure those assets get served by WebpackDevServer.// When you `import` an asset, you get its (virtual) filename.// In production, they would get copied to the `build` folder.// This loader doesn't use a "test" so it will catch all modules// that fall through the other loaders.{// Exclude `js` files to keep "css" loader working as it injects// its runtime that would otherwise be processed through "file" loader.// Also exclude `html` and `json` extensions so they get processed// by webpacks internal loaders.// PLEASE NOTE THE .wasm REGEX HERE!!!exclude:[/\.(js|mjs|jsx|ts|tsx)$/,/\.html$/,/\.json$/,/\.wasm$/],loader:require.resolve('file-loader'),options:{name:'static/media/[name].[hash:8].[ext]',},}
If you found those notes useful, or you want to suggest something else, please tell me onTwitter, or consider offering me aKo-fi !
A new version of ScreenOn is now available!
ScreenOn is an application to measure and evaluate how much are you using your smart phone.
The main change in this version is a revamp of the "distribution view", as seen in the screenshot below.
I decided to replace the ugly scatter chart of the previous version: it was hard to read,
and the "logarithmic/normal" stuff was annoying and unclear.
This new chart separates usage intervals into several classes. It lets the user understand some patterns
of its phone usage: a lot of very short changes may mean they were anxious, and several long sessions
(over 5 minutes / 300 seconds) may mean that they spend a lot of time reading on their phone, scrolling timelines/feeds or
watching videos.
This version also features some bug fixes in the data manipulation functions,
and handles the "no data" case better. There is some work left to do - we have to teach new users
how to feed data to the application from the Automate applet - but it's a first step nonetheless!
We also have an Hacktoberfest contribution! A new contributor
integrated ScreenOn with TravisCI, a platform for continuous integration.
It will help us understand if some new code introduces regressions or does not build,
catching annoying problems before they have a chance to slip into releases!
(frontend) automatically load data from a week ago to today
(frontend) handle "no data" case in distribution view and daily usage view
(frontend) write tests for client-side data manipulation functions
(frontend) rework distribution view: replace scatter chart with horizontal
bar chart to show classes of usage (under 5 seconds, 5-10 seconds, 10-30 seconds,
30 seconds to one minute, 1-5 minutes, over 5 minutes)
If you want to test ScreenOn, read its source code and contribute bugfixes... head over toGithub! You can alsohelp this project by offering me aKo-Fi
px-vis-timeseries is one of the most used web components in the Predix
toolkit set. This must-know component is essential, as it lets
chart time-based data and events! Unfortunately, it's not so
easy to configure it to show the correct timezone.
Let's see how to set it right, then.
Disclaimer: in this post, we are discussing px-vis-timeseries, version 4.0.0.This post may not apply to future versions of the components discussed here.
Gotcha #1: the default x-axis timezone is UTC
The default timezone of the X axis is UTC. While it may be the norm in
engineering disciplines or technical users, it may confuse normal users:
they are used to the timezone they live in, and UTC may confuse them (if they're
even aware of it!)
To see the time in the browser's timezone, you need to set x-axis-type="timeLocal".
By default, it's set to x-axis-type="time".
Gotcha #2: tooltip and register timezones can be configured separately!
The tooltip and the register are independent components, and px-vis-timeseries
does not configure them automatically, nor offers a single configuration point.
While this design choice makes the parent implementation simpler and
allows for more flexibility, the users have to remember to configure each
sub-component.
The configuration can be provided by setting tooltip-config
and register-config, to set the tooltip and the register.
To set the timezone, you can either use a timezone string (such as "Europe/Rome") or use
the browser-provided Intl global object to retrieve the current timezone.
I've prepared a minimal proof of concept to experiment with.
This PoC is hosted on wintermade.it: it lets you test the code for both gotchas, all in one component.
If you want to modify it, there is a live-editable version on Glitch, a code editing service that lets you host your experiments/small websites.
In both cases, you need to enable javascript to see the example :D.
Did these notes save your day? Offer me aKo-Fi! Or drop me atweetif you find errors or typos!
Some time ago, I was a bit nervous, so I started to browse twitter on my phone.
I didn't start using it because I received a notification or a friend sent me
an interesting post, but to divert my attention to the task I should have been working on.
After some minutes spent scrolling the timeline, it seemed to me that I "woke up"
and understood what I was doing. I understood that it was not the
first time it happened, but it was just the last occurrence of a long
series of "procrastination attack".
I needed to stop wasting my time reading rants of randos on the internet.
Before working on a solution, though, I wanted to know how much time I actually was spending
on the phone.
So, I started working, and created ScreenOn. The solution is composed of three parts:
an Automate flow to read screen status and save every change to a local sqlite3 database
an Automate flow to push screen changes from sqlite3 to a Python web application
a web application (Python+javascript) to store and see visualizations/stats
Automate is an Android application that lets you develop your own programs
via a visual language. To create a program, you add, configure and connect
blocks, and each block corresponds to a single action or condition.
In a single program, you can create different flows, that are executed
in parallel.
As you can see, there are different flows:
Start: detect screen status changes (when it turns on or off), and save them
to an sqlite database
Backup: save screen changes records to a Python webservice on the same network
Config: configures the database
When the screen turns on or off, the applet saves the current time as a unix timestamp,
and the status of the screen (1 = on, 0 = off).
Web application
ScreenOn has a web application, with a Polymer frontend
The data are provided by the Python webservice that is also invoked
by the Backup Automate flow.
The web application lets us interact with some charts, so let's see them!
Usage per Day is the default view: for each day in the selected period,
it shows the time spent on the phone and how many times the screen
was turned on and off that day.
Screen Changes is a cumulative chart of the phone usage time
in the selected day, and how many changes triggered the changes.
The "screen status changes" line should help users understand
short-timed phone usages.
Distribution of Usage aims to highlight the duration of
the usage, using a xy-scatter chart. On the x-axis,
we can find the interval of time when the phone screen
was turned on. On the y-axis, we can find how many
times that interval "happened".
There are two "modes": the default one (Logarithmic)
aims to highlight the short-timed usages, by applying
the log operation to the time intervals. This operation
compresses the larger values, while leaving more space
to the smaller values.
The other one, Normal does not apply any compression
to the data, and is better suited when we need
to analyze longer time intervals.
Analysis
This is the daily usage chart, from April to June 2018.
We can see that the behavior is more or less the same every week:
on work days (from Monday to Friday) I use the phone less than
during the week-end (Saturday and Sunday). The only exception is
25th of April, that is a national holiday in Italy.
This is an example of a Wednesday, a normal work day.
There are some things I can show you:
My work hours are from 9 AM to 6 PM, so you can see
that I don't use the phone very much during that time.
The increase near 2 PM is my lunch break: I scroll Twitter, check my private email
account, and show small videos to friends.
At the time, I was using Youtube as my music player when on the road.
It was before the Premium stuff, so you had to keep your screen on even if
the phone was in the pocket and you weren't looking at it.
Every Wednesday evening, a streamer I follow
on Twitch.tv streams a lore walk-through of the whole Metal Gear Solid saga,
and every stream is ~3 hours long...
This screenshot will help us see how the Screen Changes chart
looks like when there are a lot of small changes.
As we can see, between 11 AM and 1 PM I used the phone several times in a row
for short period of times. We can understand that by seeing that the black
line (the time the screen was turned on) grows slowly, but the orange line
(the number of changes) grows very fast in the same interval.
Next Steps
No software is ever finished. ScreenOn can benefit from additional work.
Some of the next steps are purely technical, others aim to capture more
data in order to better understand why the user is using their phone
and how they're spending their time.
1) CSV Exports
Sure, an user could just stop the web application, dump the database
and restart the application, but why should they do that?
What if you want to perform some other kind of analysis on the data?
I want to empower the users and let them create their own analyses.
2) Handle timezones better
Right now, the Automate flow that records the screen changes only records an unix timestamp.
It may works in a narrow set of conditions, such as the phone never changes the timezone
and the browser is in the same timezone of the phone.
ScreenOn can do better: the user should be able to set a preferred timezone, or set
the chart to use data's timezone!
3) Twitch/Youtube is not Twitter/Mastodon scrolling!
In the analysis, we saw that the phone was used for long period of times during evenings and nights.
Personally, I prefer to turn on Youtube or Twitch on my mobile device, either as TV-like device or
to actively see my favourite content creators on those platforms.
I don't know how to implement it yet, but it should not be very difficult.
4) Ask why you are using the phone, if using it for more that X minutes or X times in an hour
There is a reason if someone is using their phone so much.
Maybe they're stuck waiting in traffic, or waiting for the build to finish.
What if they just could be anxious or don't want to think about something that makes them nervous?
If this is the case, "setting a goal to use the phone less" is just treating the symptoms
instead of curing the disease: the phone became an anxiety-relief device, nervous people
will use something else instead.
A tracker like ScreenOn cannot register context, so the Automate applet may
periodically ask the user how they feel and why are they using the phone.
If you want to propose a new kind of chart, or point out some errors and typos, don'thesitate and let me know! Contact me onTwitter.
If you liked this article, why don't you offer me aKo-Fi?