❝ Jane Austen, line by line every 15 minutes.
In 2017 I needed a challenge. From this germ of an idea @pureausten was born.
Inspiration
I looked to "The Bard" for inspiration. [0] The twitter feed @iam_shakespear, live tweets the entire works of Shakespeare, one line at a time every ten minutes. All one hundred and twenty thousand lines of the Bard. Impressive.
Limitations
The problems tweeting a book on twitter are numerous. The core challenges: Find a collection of books that an audience might read; Break them down into one hundred and twenty characters chunks; Then tweet at regular intervals. How do I tweet without a keyboard? [1]
Objectives
The key objectives I want to achieve through this Art enabled technology is brevity [2] and culture. Culture isn't something you can easily create or enforce with technology and everybody deserves the right to read in peace. The Internet can be a hostile environment for people, so the idea of ^social programming^ to create a a positive culture is a priority.
Secondary objectives I will try to achieve:
- Be regular and reliable
- Be free from annoying distractions.
- Have zero tolerance of abuse, jerks and dickheads [3]
- Regular messaging of updates or technical problems
- Transmit an authors entire catalogue
- Reply in third person impersonal [4]
- Limit following readers
- Have fun
Who will read @pureausten if it's not fun?
The Boring Technical Bits
❝ Skip this section if you don't like computers.
Prior to starting, I needed to consider the technical infrastructure: networking, computing and software.
Networking
❝ 24/7 Internet access (Easy)
Access to the Internet via my 24/7 NBN router. I have the hardware wired to a router to the Internet. Networking is independent from both computing and software. If the power drops, I can resume on a back-up system using my laptop and mobile phone.
Computer
❝ A computer that is cheap, uses little power and is reliable (Easy)
I need access to a computer with:
- Low cost
- Low power requirement
- Cheap and reliable data storage
- Uses open source / free software infrastructure
- Networking using Cat-5 cable
- Reliabile operation
My choice in computing is the commodity RaspberryPi (RPi). I chose the RPi 3B with the BCM2837B0 chipset. [5],[6]
- Cost: AUD $50 / GBP £35
- Version: RPi/3B
- Power: Very low power requirements (5V DC, 3 Amps).
- Processor: Very fast 64 bit, A8 4xCore, running at 1.4GHz.
- Memory: 1Gb RAM.
- Software infrastructure: Rasbian Operating System.
- Reliability: Low power requirements, no fan required thus able to run the machine 24 hours a day, seven days a week.
- Storage: Commodity SD cards.
- Networking: Gigabyte Ethernet (cable), Wireless Bluetooth (WiFi)
- Hackability: 2 x USB 2.0 and USB 3.0 ports
The RPi is the bees-knees of computing. Cheap, fast and programmable. Exactly what's required. Did I mention it's cheap?
Software
❝ I need a piece of software that parses books into tweets (Hard)
I need to create software that inputs a text file, processes that file into lines less than one hundred and twenty characters. I need another piece of software that tweets lines to twitter. Finally I need code to use as a timer to tweet at regular time intervals.
The following is required:
1 Reading: Input a book
2 Parsing: Process book into single lines of text
3 Communication: Tweet exactly one line
4 Timing: Tweet one line of text at fixed time intervals
5 When a book finishes, repeat the entire process.
Simple isn't it?
I started work on 17th August 2017 taking about a week of work to complete. Apart from some minor adjustments, no changes to code has been made since 2017.
I found step two the most difficult. Step two reads a single book from a file then shreds (parses) the book, paragraph by paragraph into lines. Below is a simplified description of process. A more detailed explanation can be found in Table One. [7]
a) STEP 1: Read book from a text file: 2 python files 1/3Kb
b) STEP 2: Process book paragraph by paragraph into a file of single lines: 7 python files, a shell script 1.5Kb
c) STEP 3: Tweet exactly one line, repeat until end of file: 2 python files, a shell script
d) STEP 4: At a fixed time interval, tweet a single line, one shell script
e) STEP 5: New book setup is a manual process
Formatting a book into a file of single lines, regardless of the line length, is balancing act between brevity and readability. Most problems with the current software can be found in STEP 2 and this is where I will focus effort into the next major update.
Parsing 101
❝ Parse: intransitive verb. Computers. To analyze or separate (input, for example) into more easily processed components.
When I parse the text of a book, I first break the body of the text into paragraphs. Then I chop up (parse) each paragraph into words and punctuation. Let's take a look at the first two parpagraphs of "Pride and Predjudice".
❝ It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.
❝ However little known the feelings or views of such a man may be on his first entering a neighbourhood, this truth is so well fixed in the minds of the surrounding families, that he is considered as the rightful property of some one or other of their daughters.
Imagine the parser chopping the paragraph into words and placing them into individual boxs. This is what the result looks like.
['It', 'is', 'a', 'truth', 'universally', 'acknowledged,', 'that', 'a', 'single', 'man', 'in', 'possession', 'of', 'a', 'good', 'fortune,', 'must', 'be', 'in', 'want', 'of', 'a', 'wife.']
The second step is to build a new line of text, one word at a time. Let us try this with the first paragraph, counting the number of characters that make up the sentence at the same time.
Char Line
Count
-------------------------------------
002 It
005 It is
007 It is a
013 It is a truth
025 It is a truth universally
... ...
... boring computing stuff
... ...
-------------------------------------
117 It is a universally acknowledged
that a single man in possesion of
a good fortune, must be in want
of a wife.
-------------------------------------Do you recognise this line?
Notice the line is less than the one hundred and twenty characters in length? We can use this line as it is. Let's repeat the process with the second paragraph?
['However', 'little', 'known', 'the', 'feelings', 'or', 'views', 'of', 'such', 'a', 'man', 'may', 'be', 'on', 'his', 'first', 'entering', 'a', 'neighbourhood,', 'this', 'truth', 'is', 'so', 'well', 'fixed', 'in', 'the', 'minds', 'of', 'the', 'surrounding', 'families,', 'that', 'he', 'is', 'considered', 'as', 'the', 'rightful', 'property', 'of', 'some', 'one', 'or', 'other', 'of', 'their', 'daughters.']
Look at all these boxes. Do you think this paragraph has more than one hundred and twenty characters in it? The character count for the second paragraph is two hundred and thirteen. [8] When I wrote the software for @pureausten, the maximum length of a line was one hundred and twenty characters, so if I naively break the paragraph at one hundred and twenty characters characters I get:
❝ However little known the feelings or views of such a man may be on his first entering a neighbourhood, this truth is so well fixed in the minds of the
I can do better. What if I look at the punctuation as well? If I parse the punctuation I will be able to clip the sentence at a punctution mark, thus resulting in greater readability and a character count less than one hundred and twenty.
❝ However little known the feelings or views of such a man may be on his first entering a neighbourhood,
This is a simplistic example of the kinds of problems you face trying to parse large passages of text into readable tweets. Now you might say, "Why can't we just have 240 characters?" Technically you can. In my experience you get longer, harder to read passages of text and still run into problems. [9]
Launch
❝ How did I come up with the handle?
I've been a twitter user for a long time, [10] I know what makes an easy to use handle. I wanted the authors name somewhere in the handle and I also want some way to distinguish my service from other Austen focused sites.
A twitter handle like @SupercalifragilisticexpialidociousAusten is less effective than a short pithy one. I originally tried @QuintessentialAusten. Can you imagine typing that name every time you wanted to read Austen? I can. I'd curse every time I tried to spell ^Quintessential^. I settled on Pure.
@pureausten was launched on 23rd July, 2017. [11]
Authors
❝ Austen wasn't my first pick.
I didn't start using authors. My first effort involved live tweeting an ancestor going to war in WW1 in the Dardenelles. [12] Austen wasn't my first pick. When I stated looking for authors, Kipling, London and Vern came to mind first. I'm a fan of JFK [13] so I started, tweeting historic speechs of John F. Kennedy. I chose Austen on a whimsy as I started the first run of Kipling and War of the Worlds.
Now I was running four sites, one manually the others on automatic:
- @SapperRenshaw: live tweeting an ancestor going to war in WW1.
- A defunct Jules Vern site for Science Fiction.
- @JFKSpoken: historic speaches of JFK.
- @PureAusten: the novels of Jane Austen.
It was chaos, so I pruned Verne, stopped JFK and concentrated on @pureausten.
Future
❝ Free, readable and fun
I want to spend my future with @pureausten improving the quality of the text being tweeted.
Bug Fix One
What will the bug fixes do? Well maybe I'll fix the section of code that chops a line with a period. Thus making titles such as "Mr." a terminator instead of continuing "Mr. Darcy ..." as you would expect. Here is an example:
Bug Fix Two
Here is another small but annoying example. Multiple spaces in a line of text.
Can you see it? There is a space before the period. Annoying, trivial and fixable. Have you seen the "Pure Austen Technical Difficulties Test-Pattern" before?
Bug Fix Three
The Pure Austen "Technical Difficulties Test-Pattern" appears when twitter tells me there is a duplicate tweet. Most times this is reported as a HTTP 403 error. It is annoying because tweeting halts, sometimes for hours before I notice. Fixing this defect is a priority.
Questions?
If you have any questions contact me on twitter at peterrenshaw or by email me at quintessential at seldomlogical com.
Reference
[0] William Shakespeare, 'This was long thought to be the only portrait of William Shakespeare that had any claim to have been painted from life, until another possible life portrait, the Cobbe portrait, was revealed in 2009. The portrait is known as the 'Chandos portrait' after a previous owner, James Brydges, 1st Duke of Chandos. It was the first portrait to be acquired by the National Portrait Gallery in 1856. The artist may be by a painter called John Taylor who was an important member of the Painter-Stainers' Company.'
The complete works of William Shakespeare are tweeted from a twitter account @iam_shakespear and has served as an inspiration and template for @pureausten.
[1] More problems? Where do I find books in a digital form? What about copyright? How difficult is it to break down a book into legible tweets? At the time I wrote the code twitter imposed a one hundred and twenty characters was the tweet limit. This impacts the choice of computing hardware.
[2] Brev•i•ty n. Concise expression; terseness.
[Last Accessed Wednesday first January, 2020]
[3] Dick•head n. vulgar, colloquial, pejorative, coarse A jerk; a mean or rude person.
[Last Accessed Wednesday 8th January, 2020]
[4] This isn't about me, it's about the subject matter.
[5] RaspberryPi. A cheap and reliable piece of RISC computer hardware, utilising the latest system on a chip (SOC) design found in modern mobile phones that are reliable, low cost and have very low power requirements. The new version four RaspberryPi machines are even more powerful, however they consume more power and may require cooling measures. That is one reason I've stuck with earlier releases.
[Last Accessed Monday 6th January 2020]
raspberrypi.org/products/raspberry-pi-4-model-b/specifications
[6] And will not be upgrading any time soon. The latest machine is higher specification but with much higher power and cooling requirements. It is also double the price. You can read more about the RaspberryPi BCM2837B0 chipset / hardware below.
[Last Accessed Monday 8th January 2020]
raspberrypi.org/blog/raspberry-pi-3-model-bplus-sale-now-35
raspberrypi.org/documentation/hardware/raspberrypi/bcm2837b0/README.md
raspberrypi.org/documentation/hardware/raspberrypi/bcm2837/README.md
[7] Table One A break down of software created to process files for @pureausten, by size, number of lines, number of characters, the function of the file and computer language used. Describes software created to process files for @pureausten, broken down by size, number of lines, number of characters, the function of the file and finally the computer language used.
- Size: the size of the source file code in Kilobytes
- Line: the number of lines in the source code
- Characters: the number of characters in the source code
- Function: description of the source code function
- Type: computer language file extension type
- py: Python 3.N
- sh: Bash 3.2
Key explanation: A break down of sofware to read a file, process it, then tweet the line at a regular interval.
==============================================
Size Line Char. Function Type
----------------------------------------------
STEP 1
Input a book
----------------------------------------------
016 039 00400 initialise py
319 954 10638 process py
STEP 2
Process a book into single
lines of 120 characters
----------------------------------------------
035 0091 00843 setup py
285 0801 07455 configure py
118 0337 03088 prepare book py
193 0507 05683 punctuation py
372 1245 11345 extraction line py
402 1154 12664 parse book py
118 0337 03078 parse book py
012 0025 00152 parse book sh
STEP 3
Tweet exactly one line
----------------------------------------------
22 054 0535 verify conf py
57 178 1692 verify py
43 095 0865 tweet manually sh
STEP 4
At a fixed time interval
tweet a single line
----------------------------------------------
21 57 351 timer sh
TOTAL
Size Kb Line Characters Type
==============================================
2013 5874 58789 14 files
============================================== [8] Where do you think one hundred and twenty characters stops at?
Char. Word
Len
------------------------------
007 However
006 little
005 known
003 the
008 feelings
002 or
005 views
002 of
004 such
001 a
003 man
003 may
002 be
002 on
003 his
005 first
008 entering
001 a
014 neighbourhood,
004 this
005 truth
002 is
002 so
004 well
005 fixed
002 in
003 the
005 minds
002 of
003 the
======== 120 characters ========
011 surrounding
009 families,
004 that
002 he
002 is
010 considered
002 as
003 the
008 rightful
008 property
002 of
004 some
003 one
002 or
005 other
002 of
005 their
010 daughters.
------------------------------
213 TOTAL
------------------------------[9] If you are really observant, you will notice that I have not parsed the punctuation. I do this for simplicity of discussion.
[10] From the launch date on 23rd July 2017 to 1st January 2020 the software behind @pureausten has tweeted seventy six thousand times.
There have been failures, but not many in the software operation. Failure mostly occurs with the interaction with the twitter API. The 403 error (Duplicate tweet) is mostly to blame and is something I'm thinking how to prevent.
[11] A twitter user since November 2006.
[12] While WW1 started in 1914 and finished in 1918, the AIF paper trail for Alfred continued till 1927. The effort continues. Sapper Renshaw (@sapperrenshaw) traces an ancestor, the first Australian Born Renshaw, Albert Edward RENSHAW from quiting his day job at his fathers Engineering firm in Melbourne, to his mortal wounding on the shore of ANZAC COVE on what we now know as ANZAC Day. The idea? Live tweet using historical records.
[13] There is something in the cadence of Kennedys speaches that lends itself to shredding. The sentences contain short but complex ideas, formed one line at a time. It reminds me a lot of Austen.
