Chris Winters Blog2022-11-25T22:05:25+00:00http://www.cwinters.com/Chris Winterschris@cwinters.comThis American Life: Fermi's Paradox and Loneliness2017-06-12T00:00:00+00:00http://cwinters.github.com/2017/06/12/this-american-life-loneliness<p><a href="https://en.wikipedia.org/wiki/Fermi_paradox">“Fermi’s Paradox”</a> is about
intelligent life in the universe – if it’s so vast and has been around so
long, why haven’t we heard from them?</p>
<p>The <a href="https://www.thisamericanlife.org/radio-archives/episode/617/fermis-paradox">recent This American Life show by the same name</a>
is a great one. And like so many of their great shows uses this basic idea as a
springboard into much more emotionally resonant stories. (At least they were
for me, YMMV.)</p>
<p>The first part of the show goes through the actual issues around the paradox –
maybe it’s because they just haven’t found us yet? Maybe it’s because they
spend more time listening and less time transmitting? Maybe we <em>have</em> heard
from them but we don’t know it? Maybe they’re living with us now! David
(Kestenbaum) got his Ph.D. in physics and talks to a couple of his old
professors about his struggle. Because the idea that we’re alone in the
universe, that this is it, really bums him out.</p>
<p>That part of the show was well done and thoughtful, but the next two are were
it really got me. They both extend the nature of the show by talking about
loneliness among people, rather than loneliness among the universe, and IMO you
should not listen to them unless you’re prepared for some emotional overflow.</p>
<p>The first of the two is loneliness in the context of a marriage. It references
a podcast (of course!) called Something Something Something, which is a
marriage therapist who records conversations with her clients and, with their
permission, broadcasts and discusses them. The anecdote is around an older
couple (in their early 60s) who have been married something like 35 years. And
for a significant amount of that time (over 20 years) the husband has been
cheating on his wife. (He says it’s sex without love, but you never know, do
you?)</p>
<p>It’s heartbreaking to hear what people who love each other can do to each
other, and how powerful inertia combined with childhood trauma can be.
Surpisingly, the wife wants to stay with her husband. And the therapist’s main
job is to get the husband to realize that his (legitimate) shame and grief is
crowding out his wife’s, that it’s not sufficient to express contrition but he
has to give her space to be angry, and even angry without a point or hope of
resolution.</p>
<p>And during the session you hear him <em>get</em> this, but then you hear him react to
something she’s said in the same old way, where his shame is preventing him
from hearing what she’s saying and from thinking about what she needs. They
point it out and he <em>gets</em> it, again, and then you realize that life isn’t a
movie and it’s not a switch you can flip. That emotional trail he implicitly
blazed over decades through his actions won’t be so easily shortcut.</p>
<p>If the first of the two pulls your spousal heartstrings the second will do the
same for your parental heartstrings. It starts as kind of a standard story,
putting the wondrous curiosity of a kid up on a pedestal: “Once she got to be
age 9 she asked all these big questions: ‘What is time? What happens after we
die? Is heaven another planet? What is love?’” Oh, that precicious kid.</p>
<p>And the dad, a professor, starts talking about all the effort he was making to
answer these questions. He’d write pages and pages and reference philosophy and
history. Oh, what a great dad.</p>
<p>Except.</p>
<p>Except what the kid wasn’t looking for answers. She was just looking for a way
to connect with her dad. And she figured that big, open questions would be a
great way for them to sit down together and talk. Due to split custody he only
got to see her so many days a week, so many weeks a year. She was going to a
new school and didn’t know anyone and she was just lonely.</p>
<p>So just like the cheating husband in the previous segment the dad was walking
down the Good Parent path that so many things in our culture blaze for us.
Sitting at a table and talking doesn’t make for a good Instagram post,
making stuff does. But that doesn’t mean we shouldn’t do it.</p>
RIP Gwen Ifill2016-11-23T00:00:00+00:00http://cwinters.github.com/2016/11/23/rip-gwen-ifill<p>Every time I saw Gwen Ifill on PBS or moderating a debate I was impressed by
Gwen Ifill. She seemed to embody the best of journalism – uncowed by power,
eyes on the prize of truth, not settling for either the easy questions or
brush-off answers.</p>
<p><a href="http://www.newyorker.com/news/daily-comment/talking-to-young-people-about-trump-with-lessons-from-gwen-ifill">This is a lovely article</a>
about her, drawing connections between her career and young people today who
can lack the context of history and the understanding that people are not
simply collections of positions and biases.</p>
<p>In particular, the last sentence here is something I aspire to.</p>
<blockquote>
<p>Gwen and I both loved clothes, but we loved the armor we wore even before we
put on a stitch, the armor we wore as we traversed roads not taken by women who
looked like us; Gwen took that moral armor into her work as a professional
journalist, just as she took her consciousness of race and racism into whatever
newsroom she inhabited, unabashedly but gently providing good information that
helped cause hateful words or hateful glances to fall to the ground and dry up.</p>
</blockquote>
Recipe for using sqitch on Codeship2015-10-03T00:00:00+00:00http://cwinters.github.com/2015/10/03/recipe-for-sqitch-on-codeship<p>The <a href="http://sqitch.org">sqitch</a> tool is a standaone database migration tool not
tied to any language. It doesn’t try and sync models with your database, or
assume that it should be managing everything in your database, or generally
stick its fingers where they don’t belong.</p>
<p>David Wheeler (the author) has put a ton of work into making installation
straightforward, as evidenced by the substantial list of
installation alternatives (<code class="language-plaintext highlighter-rouge">cpan</code> and <code class="language-plaintext highlighter-rouge">cpanm</code>, <code class="language-plaintext highlighter-rouge">apt-get</code>, <code class="language-plaintext highlighter-rouge">homebrew</code>, <code class="language-plaintext highlighter-rouge">yum</code>) on
the homepage.</p>
<p>However the linux-based ones assume you’ve got <code class="language-plaintext highlighter-rouge">sudo</code> rights, and that’s not
true on CI services like Codeship. Here are the steps I went through to make
<code class="language-plaintext highlighter-rouge">sqitch</code> available for my tests:</p>
<p><strong>1. Test settings</strong></p>
<p>Most of the work is done here.</p>
<p>The first step is to download and install
<a href="http://search.cpan.org/dist/App-cpanminus/">cpanminus</a> because the default
<code class="language-plaintext highlighter-rouge">cpan</code> isn’t as amenable to one-time automated use.
<sup id="sqitch-fnref:1"><a href="#sqitch-fn:1" rel="footnote">1</a></sup></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pushd ~/bin
curl -L https://cpanmin.us/ -o cpanm
chmod +x cpanm
popd
</code></pre></div></div>
<p>Yes, this is downloading a script directly from the internet. It’s one that’s
auto-generated from the module’s source, and the author is very well known and
regarded. If we were paranoid we could download the script ourselves and put it
on a server we control.</p>
<p>Next we need to install <code class="language-plaintext highlighter-rouge">sqitch</code>. We’re using Postgres so in addition to
installing the module itself we’re installing the Perl-to-Postgres database
interface:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/bin/cpanm --notest App::Sqitch DBI DBD::Pg
</code></pre></div></div>
<p>Finally we need to modify the path and tell Perl where these additional
libraries are stored. The output of the above step starts with this banner:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>! Can't write to /usr/local/share/perl/5.18.2 and /usr/local/bin: Installing modules to /home/rof/perl5
! To turn off this warning, you have to do one of the following:
! - run me as a root or with --sudo option (to install to /usr/local/share/perl/5.18.2 and /usr/local/bin)
! - Configure local::lib your existing local::lib in this shell to set PERL_MM_OPT etc.
! - Install local::lib by running the following commands
!
! cpanm --local-lib=~/perl5 local::lib && eval $(perl -I ~/perl5/lib/perl5/ -Mlocal::lib)
</code></pre></div></div>
<p>The very end of that first line tells us the path where our modules will be
installed. (It’s Codeship-specific, but AFAIK it’s stable from
project-to-project.) We’ll need to include that in the path and library
setting. With those environment variables here’s what the full test settings
look like (in an easy copy-paste block):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># install cpanminus
mkdir -p ~/bin
pushd ~/bin
curl -L https://cpanmin.us/ -o cpanm
chmod +x cpanm
popd
# install sqitch and postgres interface
~/bin/cpanm --notest App::Sqitch DBI DBD::Pg
# make sqitch and libraries available
export PATH=/home/rof/perl5/bin:$PATH
export PERL5LIB=/home/rof/perl5/lib/perl5
</code></pre></div></div>
<p><strong>2. Your test script</strong></p>
<p>We have a standard <code class="language-plaintext highlighter-rouge">test.sh</code> script at the root of every project. It doesn’t do
a whole lot, but it gives us a place to put CI-specific configuration. In the
Codeship case the only thing we need to change is the port.
<sup id="sqitch-fnref:2"><a href="#sqitch-fn:2" rel="footnote">2</a></sup>
Since we’re using 9.4-specific features we set it to 5434. Here’s the setup we
do to create a database and deploy migrations with sqitch:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#!/bin/bash
DATABASE_NAME="myproject_test"
PORT=5432
if [ "$CI" = "true" ]; then
PORT=5434
fi
DB_COUNT=`psql --port $PORT -l | grep -c "$DATABASE_NAME "`
if [ $DB_COUNT -eq 0 ]; then
echo "Database does not exist, creating..."
psql --port $PORT -c "CREATE DATABASE $DATABASE_NAME" postgres
fi
sqitch deploy db:pg://localhost:$PORT/$DATABASE_NAME
if [ $? -ne 0 ]; then
echo "Sqitch migration FAILED. Refusing to run tests."
echo "Dropping database for clean next run."
psql --port $PORT -c "DROP DATABASE $DATABASE_NAME" postgres
exit 1
fi
echo "Database setup done. Run tests..."
...
</code></pre></div></div>
<p><strong>3. Profit!</strong></p>
<p>A successful test output now starts with something like this
(individual migration names depend on your project, of course):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Database does not exist, creating...
CREATE DATABASE
Adding registry tables to db:pg://localhost:5434/myproject_test
Deploying changes to db:pg://localhost:5434/myproject_test
+ ltree ........ ok
+ groups ....... ok
+ memberships .. ok
+ activities ... ok
Database setup done. Run tests...
</code></pre></div></div>
<p>Go forth and migrate!</p>
<div class="footnotes">
<ol>
<li class="footnote" id="sqitch-fn:1">
There are many tips -- like
<a href="http://stackoverflow.com/questions/3462058/how-do-i-automate-cpan-configuration">this one from SO</a>
-- on generating an automation-friendly config for CPAN, but
they seem to be geared toward making tasks automatable for
convenience rather than necessity.
<a href="#sqitch-fnref:1" title="return to article"></a>
</li>
<li class="footnote" id="sqitch-fn:2">
<a href="https://codeship.com/documentation/databases/postgresql/">Codeship runs</a>
different versions of Postgres on different ports, which is a pretty
elegant solution IMO.
<a href="#sqitch-fnref:2" title="return to article"></a>
</li>
</ol>
</div>
Systems win2015-03-10T00:00:00+00:00http://cwinters.github.com/2015/03/10/systems-win<p>From
<a href="https://medium.com/backchannel/a-spreadsheet-way-of-knowledge-8de60af7146e">A Spreadsheet Way of Knowledge</a>,
Steven Levy, originally published in November 1984 Harper’s:</p>
<blockquote>
<p>…The spreadsheet user has no way of quantifying a corporate tradition or
the misery of stockholders or whether the headaches of a drawn out takeover
bid will ultimately harm the corporate climates of the firms involved.</p>
<p>The flexibility of spreadsheets can encourage other heartless moves from
headquarters. It is no great drain on an executive’s time to experiment with
all sorts of odd, even insidious. He might ask “What if we dropped our
pension plan?” Then he might run his idea through a spreadsheet and find a
huge gain in capital- and there would be an unthinkable, in hard figures.</p>
</blockquote>
<p>From
<a href="http://www.amazon.com/Close-Machine-Technophilia-its-Discontents-ebook/dp/B007FU83DY/">Close to the Machine</a>,
Ellen Ullman, 1997 (86-7)</p>
<blockquote>
<p>I remembered the first time I saw a system infect its owner… The company’s
employees had been there for ten and twenty years, particularly the women,
mostly clerical workers. They were the ones who would be most affected by the
new system, yet they went about learning it with a homey cheerfulness that
surprised me.</p>
<p>…But just after we ordered dessert, Mr. Banner leaned over to me and asked,
“Can you keep track of keystrokes?”…</p>
<p>“I don’t know offhand. But why? Why would you want to do that?”…</p>
<p>“Well, take Mary. I’d like to know everything that Mary does in a day.”</p>
<p>Mary was the receptionist and general office manager. She was [the owner’s]
oldest employee, twenty-six years. As I recalled, Mary knew every one of the
company’s clients by name. For the first several years of her employment,
when [the owner’s] kids were small, she used to pick them up from school,
take them home, and pour them milk.</p>
<p>“But why do you want to keep an eye on Mary? She’s doing very well with the
system. I mean, is there a problem?”</p>
<p>“Oh, no. No problem,” said [the owner], “but, you know … Well, I’m just
curious. All those years she’s been out running things, and now I can find
out exactly what she does.”</p>
<p>“So you want to know about Mary just because you can?” I asked….</p>
<p>“Hmm. That’s it, I suppose. The way I look at it, I’ve just spent all this
money on a system, and now I get to use it the way I’d like to.”…</p>
<p>The system was installed, it ran, and it spoke to him: you can know every
little thing you always wanted to know. You can keep an eye on the woman you
trusted to pick up your kids from kindergarten. You can count every
keystroke, and you want to count them simply because it’s possible. You own
the system, it’s your data, you have power over it; and, once the system
gives you the power, you suddenly can’t help yourself from wanting more.</p>
</blockquote>
Corollary of constant change2015-03-03T00:00:00+00:00http://cwinters.github.com/2015/03/03/corollary-of-constant-change<blockquote>
<p>We should know this, I thought. I imagined other, better experts who could do
this job more quickly. But the guilt passed. I’d been doing this long enough
to know that, pitiful as we were, we were it. The three of us struggling
through the running of this database on a new operating system, flipping
through incomplete documentation, using trial-and-error settings of options
– this was what passed for expertise in our business….</p>
<p>…The corollary of constant change is ignorance. This is not often talked
about: we computer experts barely know what we’re doing. We’re good at
fussing and figuring out. We function well in a sea of unknowns. Our
experience has only prepared us to deal with confusion. A programmer who
denies this is probably lying, or else is densely unaware of himself.</p>
</blockquote>
<p>– <a href="http://www.amazon.com/Close-Machine-Technophilia-its-Discontents-ebook/dp/B007FU83DY/">Close to the Machine</a>, Ellen Ullman (108, 110)</p>
Spreadsheets!2015-02-28T00:00:00+00:00http://cwinters.github.com/2015/02/28/spreadsheets<p>A recent <a href="">Planet Money episode</a> did a brief history of the spreadsheet, using
as its base a 1984 Harper’s article about
<a href="https://medium.com/backchannel/a-spreadsheet-way-of-knowledge-8de60af7146e">A Spreadsheet Way of Knowledge</a>.
<sup id="spreadsheet-fnref:1"><a href="#spreadsheet-fn:1" rel="footnote">1</a></sup>
I think the podcast is a great intro to the history – VisiCalc, Dan Bricklin,
people buying computers just to use it – and gives some sense of how important
the spreadsheet was to the computer being accepted in every day
life. The contrasts between doing spreadsheets manually – on paper! – and on
the computer were so stark and dramatic that the typical arguments countering
technology adoption got swept away.</p>
<p>How stark? One of the seductive features of technology is its maleability. One
aspect of this is asking it “What if?” questions, and the speed at which you
can do this can fundamentally change how you think about problems and solutions.
<sup id="spreadsheet-fnref:2"><a href="#spreadsheet-fn:2" rel="footnote">2</a></sup>
Back in the day even a simple question – for example, one they posed during
the show was something like “What happens if we make candy bars 5%
smaller?” – would take <strong>days</strong> to answer. Spreadsheets made it possible to
answer it immediately and then move on to the next question with that answer
fresh in your brain. So: <strong>stark</strong>.</p>
<p>And it so happens that I’m reading (again)
<a href="http://www.amazon.com/Close-Machine-Technophilia-its-Discontents-ebook/dp/B007FU83DY">Close to the Machine</a>
which has a great few pages (76-80) about the spreadsheet and its differences
to the Web. Ullman wrote the book in 1997: fairly early in the Web’s life and
before a lot of the interactivity that we take for granted these days, but
nearly 20 years after the spreadsheet’s introduction. But I think the
comparisons and her points continue to apply today. Generally they fall into
examples of the tension between software for consumption and production.</p>
<p>First, there’s how users interact with the technology:</p>
<blockquote>
<p>When I watch the users try the Internet, it slowly becomes clear to me that
the Net represents the ultimate dumbing down of the computer. The users seem
to believe that they are connected to some vast treasure trove – all the
knowledge of our times, an endless digitized compendium, some electronic
library of Alexandria – if only they could figure out how to search it
properly. They sit and click, and look disconcertedly at the junk that comes
back at them. Surely it must be their fault, they reason; surely if they just
followed the right links, expressed their query more accurately, used another
search tool, then pages and pages of interesting information would soon be
theirs….</p>
<p>In front of a spreadsheet, however, their helplessness and confusion vanish.
When users want to show me the sort of information they have been storing,
they open elaborate, intricate spreadsheets full of lists and macros and
mathematical formulas, links to databases, mail-merge programs, and word
processors. They have, in effect, been programming.</p>
</blockquote>
<p>A lot of programmers look down on spreadsheets. Particularly because a fairly
common programming task is turning a single-user spreadsheet into a multi-user
shared system.
<sup id="spreadsheet-fnref:3"><a href="#spreadsheet-fn:3" rel="footnote">3</a></sup>
This typically involves unwinding years of history and business
decisions codified in a spreadsheet’s workings, a hard task in any medium. But
I think it’s even more difficult because complex spreadsheets can be <em>foreign</em>
to developers: they’re not laid out like a program that you can read serially,
the functionality has to be ferreted out piece by piece.</p>
<p>But spreadsheets are indeed programming. That fact that they’re commonly
created by actual domain experts who are solving actual problems (vs creating
tools to help other people solve problems) should be reason enough to mine them
for gold.</p>
<p>Here’s another way of looking at that question:</p>
<blockquote>
<p>The spreadsheet presumes nothing. It has no specific knowledge, no data, no
steps it performs. What it offers instead is a complex vocabulary for
expressing knowledge. It is, literally, a blank sheet of paper with a notion
of columns and rows – and everything held on that sheet is presumed to come
not from the program but from the human user. In the relationship between
human and computer that underlies the spreadsheet, the human is the
repository of knowledge, the samrt agent, the active party. The user gives
data its shape – places it in columns and rows, expresses the complex
relationships among those columns and rows – and eventually turns data into
more knowledge. It is the end user who creates information, who gives <strong>form</strong>
to data, who <strong>informs</strong> the spreadsheet.</p>
</blockquote>
<p>The blank-sheetness gets at the heart of programming. Spreadsheets model the
world, just like software does.</p>
<p>Finally, another contrast to carry us out:</p>
<blockquote>
<p>The spreadsheet is the program that all but created the personal computer.
The spreadsheet and the word processor – two tools empty of information, two
little programs sitting patiently and passively for their human owners to put
something interesting into them. Now, fifteen years later, the Internet
browser is the program creating the second generation of the personal
computer. the browser – a click-click baby tool for searching the Web, where
everything of interest already resides. It is a journey through the looking
glass in the age of information: one pill makes you larger, and one pill
makes you small.</p>
</blockquote>
<p>If you haven’t read her book I strongly recommend it. It’s a collection of
essays, and a fairly slim one at that. Her book <a href="http://www.amazon.com/Bug-Ellen-Ullman-ebook/dp/B007FU8F28/">The Bug</a>
is also a fun one. She’s a wonderful evoker of many essences of software and
creating it.</p>
<div class="footnotes">
<ol>
<li class="footnote" id="spreadsheet-fn:1">
<p>
...written by Steven Levy. It's pretty amazing how long he's not only been writing, but also writing with both insight and perspective.
<a href="#spreadsheet-fnref:1" title="return to article"></a>
</p>
</li>
<li class="footnote" id="spreadsheet-fn:2">
<p>
...which is, of course, the driving motivation behind agile software
practices. And the better we get at it the more questions we can ask -- about
tools, practices, libraries, patterns, hardware, all the way down.
<a href="#spreadsheet-fnref:2" title="return to article"></a>
</p>
</li>
<li class="footnote" id="spreadsheet-fn:3">
<p>
shout out to my Summa friends
<a href="#spreadsheet-fnref:3" title="return to article"></a>
</p>
</li>
</ol></div>
Some follow up: waffles and Invisibilia2015-02-01T00:00:00+00:00http://cwinters.github.com/2015/02/01/some-follow-ups<h2 id="easier-waffles">Easier waffles</h2>
<p><a href="/2015/01/25/making-waffles.html">These waffles</a> were delicious, but required a
big time investment, and their fussiness indicates to me that they’re fairly
sensitive to variation. (8 1/2 TB of butter? So use that whole stick, then bust
open another one so you can shave off 1/16 of it for that extra another half
tablespoon. Fussy.)</p>
<p>On top of all that the cleanup was fussy too. I wound up heating up the iron
again then dredging up some of the big pools of carmelized sugar with Q-Tips.
Annoying.</p>
<p>Fortunately there are many recipes under the sun. The waffle iron came with a
few, including <a href="http://www.food.com/recipe/good-night-waffles-202843">this one</a>
that someone replicated online. It’s also a night-before recipe, but it’s one
that takes only 10 minutes and has only one rise (overnight). Here’s what the
dough looks like the next morning:</p>
<p align="center">
<a href="https://www.flickr.com/photos/cwinters/16418845095" title="Good morning waffle batter by Chris Winters, on Flickr"><img src="https://farm8.staticflickr.com/7324/16418845095_1bc6fef216_m.jpg" width="240" height="180" alt="Good morning waffle batter" align="center" /></a>
</p>
<p>They’re a more traditional batter so cooking isn’t quite so tricky either. The
results are excellent – just a little bit of sweet, and balanced between
doughy and airy. So I think the reward-to-effort ratio is higher than the other
ones, though it’s certainly possible that with practice (and maybe a stand
mixer) the effort for the Liège waffles will diminish.</p>
<p align="center">
<a href="https://www.flickr.com/photos/cwinters/16393229916" title="Good
morning waffles all cooked by Chris Winters, on Flickr"><img src="https://farm8.staticflickr.com/7441/16393229916_c1443b0b42_m.jpg" width="240" height="151" alt="Good morning waffles all cooked" /></a>
</p>
<p>Finally, part of the answer to “how do I cleanup after Liège waffles?” is:
make more waffles! Unless you have a waffle iron that can detach the plates for
cleaning, you’ll inevitably miss some of the hardened sugar. But the next
waffles you make will absorb those little bits and what you’ll have left after
that batch will be markedly cleaner. Love those types of cleanup!</p>
<h2 id="new-podcast-invisibilia">New podcast: Invisibilia</h2>
<p>A new podcast that started <a href="2014/11/06/podcasts-again.html">since my last review</a> is
<a href="http://www.npr.org/programs/invisibilia/">Invisibilia</a>. It’s set a very high
bar for itself with the first few episodes not only in production quality and
storytelling, but in presenting me with moments that make me question
fundamental things.</p>
<p>For example, the most recent episode
(<a href="http://www.npr.org/programs/invisibilia/382451600/entanglement?showDate=2015-01-30">Entanglement</a>)
talked at one point about humans’ ability to mirror one another, unconsciously.
Small things like physical mannerisms, ways of speaking, even autonomic
processes like breathing. But also large parts of our identity – how we
interact with others, our emotional states.</p>
<p>And two psychologists came to the realization that treating people is even
harder than they’d previously thought because of these entanglements. In fact
the very notion of you as an individual gets called into question because
you’re a constantly changing amalgam of your environment, as is everyone around
you.</p>
<p>There are definitely some WTF moments when they’re explaining unusual
conditions that people have, but they don’t leave it at that and dig into what
those conditions illustrate. Wonderful stuff, and something that <strong>everyone</strong>
can get something out of.</p>
Creating buckets for relative minutes in SQL2015-01-28T00:00:00+00:00http://cwinters.github.com/2015/01/28/sql-relative-minute-buckets<p>Aggregating use over buckets of time is really useful when doing performance
testing. For example, if you have a script that throws a bunch of requests at
your system in a known time series you can re-run that script to gauge the
effectiveness of your changes. So say we have a table:</p>
<figure class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">tasks</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">INT</span> <span class="k">NOT</span> <span class="k">NULL</span><span class="p">,</span>
<span class="n">task_type</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">8</span><span class="p">)</span> <span class="k">NOT</span> <span class="k">NULL</span><span class="p">,</span>
<span class="n">created</span> <span class="nb">TIMESTAMP</span> <span class="k">NOT</span> <span class="k">NULL</span><span class="p">,</span>
<span class="n">completed</span> <span class="nb">TIMESTAMP</span><span class="p">,</span>
<span class="n">operations</span> <span class="nb">INT</span><span class="p">,</span>
<span class="k">PRIMARY</span> <span class="k">KEY</span><span class="p">(</span><span class="n">id</span><span class="p">)</span>
<span class="p">);</span></code></pre></figure>
<p>and we know the tasks we’re interested in aggregating were created between
23:41 and 23:51. We can do the following in Postgres to aggregate the
operations into buckets by creation minute so we can show the average elapsed
time:</p>
<figure class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">WITH</span> <span class="n">offsets</span> <span class="k">AS</span> <span class="p">(</span>
<span class="k">SELECT</span>
<span class="p">(</span><span class="k">EXTRACT</span><span class="p">(</span><span class="k">minute</span> <span class="k">FROM</span> <span class="n">created</span><span class="p">)</span> <span class="o">||</span> <span class="s1">' minutes '</span>
<span class="o">||</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="k">second</span> <span class="k">FROM</span> <span class="n">created</span><span class="p">)</span> <span class="o">||</span> <span class="s1">' seconds'</span><span class="p">)::</span><span class="n">interval</span> <span class="k">AS</span> <span class="n">initial</span>
<span class="k">FROM</span>
<span class="n">tasks</span>
<span class="k">WHERE</span>
<span class="n">created</span> <span class="k">BETWEEN</span> <span class="s1">'2015-01-27 23:41'</span> <span class="k">AND</span> <span class="s1">'2015-01-27 23:51'</span>
<span class="k">ORDER</span> <span class="k">BY</span>
<span class="n">created</span>
<span class="k">LIMIT</span>
<span class="mi">1</span>
<span class="p">),</span>
<span class="n">summary</span> <span class="k">AS</span> <span class="p">(</span>
<span class="k">SELECT</span>
<span class="n">id</span><span class="p">,</span>
<span class="n">task_type</span><span class="p">,</span>
<span class="n">operations</span><span class="p">,</span>
<span class="n">created</span><span class="p">,</span>
<span class="k">EXTRACT</span><span class="p">(</span><span class="k">minute</span> <span class="k">FROM</span> <span class="p">(</span><span class="n">created</span> <span class="o">-</span> <span class="n">offsets</span><span class="p">.</span><span class="n">initial</span><span class="p">))</span> <span class="k">AS</span> <span class="n">created_minute</span><span class="p">,</span>
<span class="n">completed</span><span class="p">,</span>
<span class="k">EXTRACT</span><span class="p">(</span><span class="k">minute</span> <span class="k">FROM</span> <span class="p">(</span><span class="n">completed</span> <span class="o">-</span> <span class="n">offsets</span><span class="p">.</span><span class="n">initial</span><span class="p">))</span> <span class="k">AS</span> <span class="n">completed_minute</span><span class="p">,</span>
<span class="k">EXTRACT</span><span class="p">(</span><span class="n">milliseconds</span> <span class="k">FROM</span> <span class="n">completed</span> <span class="o">-</span> <span class="n">created</span><span class="p">)</span> <span class="k">AS</span> <span class="n">elapsed</span>
<span class="k">FROM</span>
<span class="n">tasks</span><span class="p">,</span> <span class="n">offsets</span>
<span class="k">WHERE</span>
<span class="n">created</span> <span class="k">BETWEEN</span> <span class="s1">'2015-01-27 23:41'</span> <span class="k">AND</span> <span class="s1">'2015-01-27 23:51'</span>
<span class="p">)</span>
<span class="k">SELECT</span>
<span class="n">created_minute</span><span class="p">,</span>
<span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="k">AS</span> <span class="n">num_created</span><span class="p">,</span>
<span class="n">ROUND</span><span class="p">(</span><span class="k">AVG</span><span class="p">(</span><span class="n">elapsed</span><span class="p">)::</span><span class="nb">numeric</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="k">AS</span> <span class="n">avg_elapsed</span>
<span class="k">FROM</span>
<span class="n">summary</span>
<span class="k">GROUP</span> <span class="k">BY</span>
<span class="n">created_minute</span>
<span class="k">ORDER</span> <span class="k">BY</span>
<span class="n">created_minute</span><span class="p">;</span></code></pre></figure>
<p>It seems long and your eyes might glaze over, but there are two interesting
parts. The first
<a href="http://www.postgresql.org/docs/9.3/static/queries-with.html">CTE</a> named
<code class="language-plaintext highlighter-rouge">offsets</code> finds the first row in our dataset and constructs a string
representing the minutes and seconds, then casts it to an interval.
So if our first row had a creation date of <code class="language-plaintext highlighter-rouge">2015-01-27 23:41:18.867530</code> the
text of the interval would be <code class="language-plaintext highlighter-rouge">41 minutes 18.867530 seconds</code>, which Postgres
will display as <code class="language-plaintext highlighter-rouge">00:41:18.867530</code>. You can test this without a table just by
extracting the interval from <code class="language-plaintext highlighter-rouge">NOW()</code>:</p>
<figure class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">SELECT</span>
<span class="p">(</span><span class="k">EXTRACT</span><span class="p">(</span><span class="k">minute</span> <span class="k">FROM</span> <span class="n">NOW</span><span class="p">())</span> <span class="o">||</span> <span class="s1">' minutes'</span>
<span class="o">||</span> <span class="s1">' '</span>
<span class="o">||</span> <span class="k">EXTRACT</span><span class="p">(</span><span class="k">second</span> <span class="k">FROM</span> <span class="n">NOW</span><span class="p">())</span> <span class="o">||</span> <span class="s1">' seconds'</span><span class="p">)::</span><span class="n">interval</span><span class="p">;</span></code></pre></figure>
<p>and get a result with something like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> interval
-----------------
00:33:56.694513
</code></pre></div></div>
<p>In the next CTE (<code class="language-plaintext highlighter-rouge">summary</code>), we’ll join to that one-row table containing an
interval to give us something to subtract from our data before extracting a
minute. So applying this interval to our first row would do:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>2015-01-27 23:41:18
- 00:41:18
===================
2015-01-27 23:00:00
</code></pre></div></div>
<p>and applying it to another row in the dataset might do:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>2015-01-27 23:48:08
- 00:41:18
===================
2015-01-27 23:06:50
</code></pre></div></div>
<p>and we’d <code class="language-plaintext highlighter-rouge">EXTRACT</code> the minute from the first row as <code class="language-plaintext highlighter-rouge">0</code>, from the other row as
<code class="language-plaintext highlighter-rouge">6</code>.</p>
<p>Finally is our actual <code class="language-plaintext highlighter-rouge">SELECT</code>: we’ve got the minute bucket and elapsed time
for each row we can do some simple aggregation (a count and average) grouping
by the bucket. Piece of cake!</p>
Making Liège Waffles2015-01-25T00:00:00+00:00http://cwinters.github.com/2015/01/25/making-waffles<p>At ModCloth we’d periodically have a themed potluck lunch. We had themes like
“Mexican” and “Pie”, but one of the unsurprisingly popular ones was
“Breakfast”. And one of my co-workers made these waffles that were just
amazing, the <a href="http://seersuckermag.com/food/read/the-best-waffle-in-america">best waffles</a>
I’d ever had in my life. By far.</p>
<p>I learned that these were called “Liège Waffles” and they were also
Belgian. They’re distinguished by having a crispier outside, due to carmelized
sugar, and by their chewiness, which I think comes from being a dough-based
rather than batter-based waffle.</p>
<p>For some reason those waffles popped into my mind lately, and I emailed the
co-worker and asked him for some tips. He pointed me at
<a href="https://liegewaffle.wordpress.com/liege-waffle-recipe-liege-gaufre-recette/">this recipe</a>
along with a few additional pointers. One waffle iron and bag of pearl sugar
later and we’re off!</p>
<h2 id="fussy">Fussy</h2>
<p>Read that recipe and keep a mental counter of the time required – it adds up,
quickly. There’s a lot of time to let the dough rise and even more to let it
rest in the fridge, so it’s not something you can do on a whim. The person who
pointed me to the recipe said to be sure to start in the early afternoon,
otherwise you’ll be up at 2 AM for steps 6-8.</p>
<p><img src="/images/blog/liege_01_after_fridge.jpg" alt="After being in the fridge overnight" /></p>
<p>While the recipe is fussy I’d say it’s not really that difficult. I was able to
do it without a stand mixer, which I’m pretty sure would’ve made the process a
lot easier. In particular step 5 results in some really thick dough, and the
cheap hand mixer I used complained mightily.</p>
<h2 id="ingredients">Ingredients</h2>
<p>The pearl sugar – I almost typed “Perl sugar” there – isn’t something you’ll
probably find in your local grocery. While I think it’s an unsubstitutable
ingredient, fortunately it’s available
<a href="http://www.wafflecabin.com/shop">on Amazon</a>
as well as <a href="http://www.wafflecabin.com/shop">the Waffle Cabin</a>
and probably other places.</p>
<p><img src="/images/blog/liege_02_after_sugar.jpg" alt="Dough with sugar, resting again" /></p>
<p>Everything else is pretty straightforward – for example, I’m reasonably sure
you don’t need <strong>Mexican</strong> vanilla extract. (You should use good stuff though,
not the imitation.)</p>
<h2 id="iron-and-temperature">Iron and temperature</h2>
<p>If you read the comments you read about people dropping a grand on a waffle
iron. The waffles that thing produces must be otherwordly. But my terra-bound
self just picked up a
<a href="http://www.amazon.com/gp/product/B00F3SA9LO/">Waring Pro WMK200</a>. It seems
pretty solid with my limited use, and when I ordered it was over half off.
(Which always makes me think a new model is imminent, but whatever.)</p>
<p>Something the recipe didn’t mention is where to put the dough on the iron. Your
waffle iron’s directions tell you how to pour <strong>batter</strong>, but not what to do
with dough. You’d figure this out eventually, but I found it was better to
place it offcenter, a little toward the rear.</p>
<p><img src="/images/blog/liege_03_offcenter.jpg" alt="A little offcenter" /></p>
<p>The reason is that when you bring down the lid you’ll pull the dough toward
you, and when the dough expands it’ll be harder to keep the lid closed if
there’s dough toward the front. And you’ll probably need to keep a hand on it
anyway to keep it closed – it’s thick dough!</p>
<p><img src="/images/blog/liege_05_keep_it_closed.jpg" alt="Keep it closed!" /></p>
<p>One of the many things the recipe demands specificity about is temperature –
between 365 and 370, ouch! Unfortunately I have neither an infrared
thermometer nor a waffle iron with degree demarcations for the heat, just
abstract numbers.</p>
<p>I didn’t find any other pointers about how these numbers (between 1 and 6) map
to temperatures. So to help others (including future me): I set it a little
above 4, though I lowered it a little for the last waffle because of the
carmelized sugar sticking to the iron.</p>
<p><img src="/images/blog/liege_04_cooked_in_iron.jpg" alt="Sticky sugar" /></p>
<p>BTW, if you have good ideas about how to clean the carmelized sugar off
<a href="http://twitter.com/cwinters">let me know</a>.</p>
<h2 id="finally">Finally</h2>
<p>And finally, the verdict – everybody loved them! I got some
good-natures teasing because it took so long to make, but nobody complained
about the results. You can eat them as-is – no syrup or anything is needed
because each waffle has so much sugar. I added some fruit, though I think
strawberries will be better in-season, especially after I’ve made this recipe a
few more times.</p>
<p><img src="/images/blog/liege_06_ready_to_eat.jpg" alt="EAT ME" /></p>
Swipyness of iPhone keyboards2015-01-24T00:00:00+00:00http://cwinters.github.com/2015/01/24/iphone-keyboards<p>One of the first things I sorely missed from my Nexus 4 was the keyboard. Which
I knew was going to happen – one of the things keeping me away from iPhones
was the lack of third party keyboards to enable swiping. That feature in iOS 8,
combined with the larger phone, wound up pulling me over.</p>
<p>Such a keyboard doesn’t come with the phone, so you need to install one. There
are loads of reviews out there looking at multiple apps that do this
(just a few: <a href="http://www.pcmag.com/article2/0,2817,2471221,00.asp">PCMag</a>,
<a href="http://www.digitaltrends.com/mobile/ios8-keyboards-confirmed/">Digital Trends</a>,
<a href="http://www.forbes.com/sites/gordonkelly/2014/09/22/ios-8-the-best-apple-keyboard-replacements/">Forbes</a>
<a href="http://www.geekwire.com/2014/6-best-new-iphone-keyboards/">GeekWire</a>,
<a href="http://venturebeat.com/2014/09/19/ios-8-keyboards-from-swiftkey-swype-fleksy-compared-everybody-wins/">VentureBeat</a>)
and of course they’re of varying helpfulness. For my purposes, someone who had
used such a keyboard on Android was a better source than someone who had built
up quick habits on the iPhone keyboard over years.</p>
<p>In addition to writeups like the above there are, of course, reviews on the App
Store. I don’t have a lot of context about reviews
(beyond <a href="http://atp.fm/">hearing folks talk about it</a>) – how much
weight to give them, etc. Overall they weren’t helpful at all, though
one part of them caught my eye. It seems like there are more people freaking
out than they would on Android because the phone can share things with a
server. Are Android folks just used to Google’s algorithms consuming everything
you do to provide a better experience? Probably not.</p>
<p>Anyway, a few things about the couple keyboards I tried.</p>
<h3 id="brilliant-keyboard">Brilliant Keyboard</h3>
<p>This had very good reviews. But the practical use was hugely frustrating. It
had a terrible time picking up intermediate letters on the swipe and the
recommendations were nonsensical. For example, trying to swipe the word
‘points’ was an exercise in futility. Not only did it only ever recognize
‘pints’, no matter how slowly I went, but it refused to offer up ‘points’ as a
suggestion.</p>
<p>To compensate I found myself swiping slower and slower, using the visual cues
(little blue dots over the keys) to ensure intermediate letters were captures.
And it was better. But the point of these keyboards is to type <strong>faster</strong>, so
slowing down just to make the app work made no sense.</p>
<p>Interface-wise, two things:</p>
<ul>
<li>I think it’s a terrible idea to take up a key for theme switching – do you
really use that more often than say, a period? Are themes really that important?</li>
<li>A trait it shares with the iPhone keyboard is not distinguishing the key
faces between upper and lower case, forcing the user to look over to the
lefthand shift indicator to tell you if you’re in upper or lower case. This
boggles my mind from Apple.</li>
</ul>
<p><img src="/images/blog/keyboard_brilliant.png" alt="Brilliant keyboard" /></p>
<h3 id="swiftkey">SwiftKey</h3>
<p>Some of the reviews referred to SwiftKey’s Android heritage so, despite the
pretty terrible reviews, I had high hopes. And they were <strong>redeemed</strong>. Not only
is there a punctuation key (‘.’, which a long-press option for five other
common ones), the keycaps change to lower case! Predictions are excellent and
smart, and they take into account intermediate letters.</p>
<p>One of the only downsides is that there’s no emoji keyset. The Android keyboard
integrates them with a long-press on return, maybe that’s an option going
forward?</p>
<p>After using this for just a couple of minutes I was hooked. We’ll see how well
it works over the long haul.</p>
<p><img src="/images/blog/keyboard_swiftkey.png" alt="SwiftKey keyboard" /></p>