Posts Tagged cocoa

Some Statistics About My Ridiculous YouTube Channel


I’ve developed a bit of a habit of recording entire concerts of musicians who don’t mindGraph their concerts being recorded, splitting them into individual songs, and uploading them to my YouTube channel with copious notes in the video descriptions. My first upload was, appropriately, the band featured in the first image on the web, Les Horribles Cernettes, singing Big Bang. I first got enough camera batteries and SD cards to record entire concerts for the K’s Choice comeback concert in Dranouter in 2009, though the playlist is short, so perhaps I didn’t actually record that entire show.

I’ve also developed a habit of going on a week-long cruise packed with about 25 days of entertainment every year, and recording 30 or so hours of that entertainment. So my YouTube channel is getting a bit ridiculous. I currently have 2723 publicly-visible videos on my channel, and 2906 total videos — the other 183 are private or unlisted, either because they’re open mic or karaoke performances from JoCo Cruise and I’m not sure I have the performer’s permission to post them, or they’re official performances that we were requested to only share with people that were there.

I’ve been wondering just how much I’ve written in my sometimes-overly-verbose video descriptions over the years, and the only way I found to download all that metadata was using the YouTube API. I tested it out by putting a URL with the right parameters in a web browser, but it’s only possible to get the data for up to 50 videos at a time, so it was clear I’d have to write some code to do it.

Late Friday evening, after uploading my last video from JoCo Cruise 2020, I set to writing a document-based CoreData SwiftUI app to download all that data. I know my way around CoreData and downloading and parsing JSON in Swift, but haven’t had many chances to try out SwiftUI, so this was a way I could quickly get the information I wanted while still learning something. I decided to only get the public videos, since that doesn’t need authentication (indeed, I had already tried it in a web browser), so it’s a bit simpler.

By about 3a.m, I had all the data, stored in a document and displayed rather simply in my app. Perhaps that was my cue to go to bed, but I was too curious. So I quickly added some code to export all the video descriptions in one text file and all the video titles in another. I had planned to count the words within the app (using enumerateSubstrings byWords or enumerateTags, of course… we’re not savages! As a linguist I know that counting words is more complicated than counting spaces.) but it was getting late and I knew I wanted the full text for other things, so I just exported the text and opened it in Pages. The verdict:

  • 2723 public videos
  • 33 465 words in video titles
  • 303 839 words in video descriptions

The next day, I wanted to create some word clouds with the data, but all the URLs in the video descriptions got in the way. I quite often link to the playlists each video is in, related videos, and where to purchase the songs being played. I added some code to remove links (using stringByReplacingMatches with an NSDataDetector with the link type, because we’re not savages! As an internet person I know that links are more complicated than any regex I’d write.) I found that Pages counts URLs as having quite a few words, so the final count is:

  • At least 4 633 links (this is just by searching for ‘http’ in the original video descriptions, like a savage, so might not match every link)
  • 267 567 words in video descriptions, once links are removed. I could almost win NaNoWriMo with the links from my video descriptions alone.

I then had my app export the publish dates of all the videos, imported them into Numbers, and created the histogram shown above. I actually learnt quite a bit about Numbers in the process, so that’s a bonus. I’ll probably do a deeper dive into the upload frequency later, with word clouds broken down by time period to show what I was uploading at any given time, but for now, here are some facts:

  • The single day when I uploaded the most publicly-visible videos was 25 December 2017, when I uploaded 34 videos — a K’s Choice concert and a Burning Hell concert in Vienna earlier that year. I’m guessing I didn’t have company for Christmas, so I just got to hang out at home watching concerts and eating inexpertly-roasted potatoes.
  • The month when I uploaded the most publicly-visible videos was April 2019. This makes sense, as I was unemployed at the time, and got back from JoCo Cruise on March 26.

So, onto the word clouds I cleaned up that data to make. I created them on wordclouds.com, because wordle has rather stagnated. Most of my video titles mention the artist name and concert venue and date, so some words end up being extremely common. This huge variation in word frequency meant I had to reduce the size from 0 all the way to -79 in order for it to be able to fit common words such as ‘Jonathan’. Wordclouds lets you choose the shape of the final word cloud, but at that scale, it ends up as the intersection of a diamond with the chosen shape, so the shape doesn’t end up being recognisable. Here it is, then, as a diamond:

titles

The video descriptions didn’t have as much variation between word frequencies, so I only had to reduce it to size -45 to fit both ‘Jonathan’ and ‘Coulton’ in it. I still don’t know whether there are other common words that didn’t fit, because the site doesn’t show that information until it’s finished, and there are so many different words that it’s still busy drawing the word cloud. Luckily I could download an image of it before that finished. Anyway, at size -45, the ‘camera’ shape I’d hoped to use isn’t quite recognisable, but I did manage a decent ‘YouTube play button’ word cloud:

descriptions

One weird fact I noticed is that I mention Paul Sabourin of Paul and Storm in video descriptions about 40% more often than I mention Storm DiCostanzo, and I include his last name three times as much. To rectify this, I wrote a song mentioning Storm’s last name a lot, to be sung to the tune of ‘Hallelujah’, because that’s what we do:

We’d like to sing of Paul and Storm.
It’s Paul we love to see perform.
The other member’s name’s the one that scans though.
So here’s to he who plays guitar;
let’s all sing out a thankful ‘Arrr!’
for Paul and Storm’s own Greg “Storm” DiCostanzo!
DiCostanzo, DiCostanzo, DiCostanzo, DiCostanzo

I’m sure I’ll download more data from the API, do some more analysis, and mine the text for haiku (if Haiku Detector even still runs — it’s been a while since I touched it!) later, but that’s enough for now!

 

, , , , , , , , ,

Leave a comment

Five of Hearts: Haiku Detector


A few weeks ago, a friend linked to Times Haiku, a website listing unintentional haiku found in The New York Times, saying ‘I’d actually pay for a script that could check for Haiku in my writings. That would make prose-production a lot more exciting! Who’s up to the script-writing-challenge?’

I knew I could do it, having written syllable-counting code for my robot choir (which I really need to create an explanation page about.) I told her I’d make it that weekend. That was last weekend, when I decided at the last moment to write an article about neutron stars and ISOLTRAP, and then chickened out of that and wrote a poem about it. So I put off the haiku program until yesterday. It was fairly quick to write, so here it is: Haiku Detector. It should work on Mac OS X 10.6 and above. Just paste or type text into the top part of the window, and any detected haiku will appear in the bottom part.

Haiku Detector looks for sentences with seventeen syllables, and then goes through the individual words and checks whether the sentence can be split after the fifth and twelfth syllables without breaking a word in half. Then it double-checks the last line still has five syllables, because sometimes the punctuation between words is pronounced. The Times Haiku-finding program has a database of syllable counts per word, but I didn’t need that since I can use the Mac OS X speech synthesis API to count the syllables. Haiku Detector makes no attempt to check for kigo (season words.)

The first place I looked for haiku was the Wikipedia page for Haiku in English. Due to the punctuation, it didn’t actually find any of the example haiku on the page, but it did find this:

Robert Spiess (Red Moon
Anthology, Red Moon Press,
1996)

How profound. Next, having declared myself contributing troubadour for New Scientist magazine, I fed this week’s feature articles through it, and found:

A pill that lowers
arousal doesn’t teach shy
people what to do

Meanwhile, there are signs
that the tide is turning in
favour of shyness.

So by 4000
years ago, the stage was set
for the next big step.

This heat makes the air
spin faster, so pulling the
storm towards the city.

Some will be cooler
and less humid — suitable
for outdoor sports, say.

The last ones seem almost seasonal.

I needed to stress-test the app with a large body of text, so I grabbed the first novel of which I had the full text handy: John Scalzi‘s Old Man’s War, which I had on my iPad on my lap to read while my code was compiling. This book has at least one intentional haiku in it, which Haiku Detector detected. Apart from that, some of my favourites are:

I hate that her last
words were “Where the hell did I
put the vanilla.”

As I said, this is
the place where she’s never been
anything but dead.

“I barely know him,
but I know enough to know
he’s an idiot.”

She’d find me again
and drag me to the altar
like she had before.

A gaper was not
long in coming; one swallow
and Susan was in.

They were nowhere to
be found, an absence subtle
and yet substantial.

And it stares at me
like it knows something truly
strange has just happened.

I haven’t got up to that fifth one in the novel yet, but it mentions a swallow, which I understand is (when accompanied by more swallows) a harbinger of Spring or Summer depending on which language you get your idioms from, so there’s the kigo.

Next I figured I should try some scientific papers — the kinds of things with words that the Times haiku finder would not have in its syllable database. You probably can’t check this unless your workplace also provides access to Physics Letters B, but I can assure you that the full text of the ISOLTRAP paper about neutron stars does not contain any detectable haiku. However, the CMS paper announcing the discovery of the boson consistent with the Higgs does:

In the endcaps, each
muon station consists of
six detection planes.

As is usual for CMS papers, the author and institute lists are about as long as the paper itself, and that’s where most of the haiku were too. Here are a few:

[102]
LHC Higgs Cross Section
Working Group, in: S.

University
of California, Davis,
Davis, USA

That’s ‘one hundred and two’ in case anyone who doesn’t say it that way was wondering.

And here are some from my own blog. I used the text from a pdf I made of it before the last JoCo Cruise Crazy, so the last few months aren’t represented:

Beds of ground cover
spread so far in front of him
they made him tired.

Apologies to
those who only understand
half of this poem.

I don’t remember
what colour he said it was,
but it was not green.

His eyes do not see
the gruesome manuscript scrawled
over the white wall.

• Lines 1 to 3 have
four syllables each, with stress
on the first and last.

(That’s not how you write a haiku!)

I don’t wear armour
and spikes to threaten you, but
to protect myself.

A single female
to perpetuate the genes
of a thousand men.

Kerblayvit is a
made-up placeholder name, and
a kerblatent cheat.

He wasn’t the first,
but he stepped on the moon soon
after Neil Armstrong.

He just imagined
that in front of him there was
a giant dunnock.

(there are plenty more where that one came from, at the bottom of the page)

She was frustrated
just trying to remember
what the thing was called.

Please don’t consider
this a failing; it is part
of your programming.

While writing this program, I discovered that that the speech API now has an easier way to count syllables, which wasn’t available when I wrote the robot choir. The methods I used to separate the text into sentences and the view I used to display the haiku are also new. Even packaging the app for distribution was different. I don’t get to write Mac software often enough these days.

Yet again, I didn’t even bother to deal out the cards because I already had something to inspire me. In my halfhearted attempt to find a matching card, I came across one about electronics in the service of ALICE, so I ran the latest instalment of Probably Never, by Alice, into it, and got this:

Or well, I have to
put up with getting called a
fake girl all the time.

The jackhole who called
me a “he/she” recognized
that he crossed the line.

If that sounds interesting, subscribe to Probably Never, and I could probably forward you the rest of that episode if you want.

And finally, two unintentional haiku from this very post:

Haiku Detector
makes no attempt to check for
kigo (season words.)

(there are plenty more
where that one came from, at the
bottom of the page)

Wait; make that three!

And finally, two
unintentional haiku
from this very post:

Have fun playing with Haiku Detector, and post any interesting haiku you find in the comments. Also, let me know of any bugs or other foibles it has; I wrote it pretty quickly, so it’s bound to have some.

I know what I’m doing for the six of hearts; I’ve planned it for a long time but still haven’t actually started it. It’s musical, so it will probably be terrible; brace yourselves. By the way, I keep forgetting to mention, but They Might Not Be Giants will be published in Offshoots 12. Yay!

, , , , , , , , , , ,

6 Comments

%d bloggers like this: