Editing Research/new digital reading experiences (section)

==''Behaviour &amp; Typography'' ==

<span id="on-screen-readability"></span>
==== On-screen readability  ====

Research on digital typography and on-screen readability provides valuable guidance for designing immersive text. For example, a [https://readymag.com/readymag/readme/tips/ Read Me! article by Readymag] highlight how factors such as eye movement, contrast, resolution, responsive layouts, and font choices influence reading comfort and comprehension on screens.

Similarly, studies on [https://www.lexend.com/ dyslexia and type design] emphasize how specific font characteristics can improve legibility for readers with diverse needs, underlining the importance of adaptable and inclusive typographic solutions in digital environments.

Very much like e-readers, the ‘immersive reading’ function, included in software such as Microsoft Word, PDF readers and Teams, allows the reader to change the column width, page background colour, line focus, and space between text according to their needs and reading style. It also allows reading text aloud, change reading speed, switch between languages and convert text.

[https://www.blinkist.com/ Blinkist] is an app that offers digital summaries of thousands of books, promising users that they can get the gist of any great book in under 15 minutes. These summaries are mostly AI generated and are available both as written text and spoken text. Unfortunately, written and spoken text are not synchronized, so they don’t work very well for Immersive reading. These summaries make the content much more approachable to a wider audience, but by design they miss a lot of the depth and nuance of the original works.

Experiments like [https://bionic-reading.com/br-apps/ Bionic Reading] demonstrate how subtle typographic interventions can guide the eye and support fluent reading. Developed by the Swiss designer Renato Casutt, Bionic Reading uses word visualization to help the brain recognize words faster. We initially recognize words by their sounds (“mama” evokes someone who cares for us) before learning to convert these sounds into written characters. Fluency in reading comes from automation, a process that is easy for some and challenging for others. Casutt’s method highlights certain letters of a word in bold, allowing the brain to fill in the rest and jump efficiently to the next set of bold letters. More information and apps can be found at Bionic Reading.

Bionic Reading works across multiple platforms, including Apple iOS and macOS, Google Android, Microsoft Windows, Google Chrome, and the web. Its interface cleans up any text and presents it in the bold-letter format. Users can adjust settings such as Fixation and Saccade (the amount of bold letters), text size, spacing, opacity, background color, and text color, with fixed presets. In the advanced paid version, readers can also select fonts from a specific collection, though the system currently works only with the Latin alphabet.

[[File:Bionic_Reading_Products_quer_02.png|300px]]

Mobile reading and fragmented text<br />
Our research focused on mobile typography, where reading increasingly happens in fragmented ways. Existing research shows that screen typography benefits from flexibility: adjustable line width, spacing, contrast, and background. As Gen Z often consumes text in fragments, combined with audio and motion, typography must remain legible while actively holding attention.

On platforms such as TikTok and Instagram, text rarely appears as continuous paragraphs. Instead, it is presented in short sequences of one to five words, revealed over time. Emphasis is created through size, colour, motion, or highlighting rather than through traditional typographic hierarchy.

(How Social media influenced fonts) https://typedrawers.com/discussion/5079/what-kind-of-impact-did-digital-social-media-have-on-fonts

<span id="text-on-social-media"></span>

=== Text on Social Media  ===

During our research we investigated two leading social media platforms that were mentioned as the most popular among our audience: Instagram and TikTok. The differences between the two platforms were substantial.

<span id="an-analysis-of-typography-in-tiktok-video"></span>
==== An Analysis of Typography in TikTok Video ====

[[File:tiktok_knowlaverse.png|250px]] [[File:tiktok_screen.bean3.png|250px]] [[File:tiktok_sussy.reddits.stories.png|250px]] [[File:tiktok_textospeechss.png|250px]] [[File:tiktok_textsconvo.png|250px]] [[File:tiktok_viralp0d.png|250px]]

The typography used in ''TikTok'' videos follows a high-impact, attention-grabbing approach designed for speed, clarity, and engagement. Here are the key elements:

'''1. All-Caps Dominance &amp; Sentence Case Variations'''<br />
Most TikTok text is set in all caps, ensuring maximum visibility and urgency. This choice aligns with how people skim content quickly. However, sentence case occasionally appears, often to differentiate between key messages and supporting text.

'''2. Bold, High-Contrast Colour Choices'''<br />
Red, yellow, black, and white dominate TikTok typography, echoing tabloid-style and meme aesthetics. These colours provide strong contrast, ensuring legibility against the fast-moving, diverse backgrounds of TikTok videos.

'''3. Random Placement &amp; No Clear Composition'''<br />
Text appears in a mix of placements—from centre to scattered unpredictably across the frame. This lack of traditional composition mirrors the chaotic, spontaneous nature of TikTok, reinforcing the platform's informal and raw aesthetic.

'''4. Sans-Serif and Monospaced Typefaces'''<br />
Sans-serif fonts (like TikTok’s default Proxima Nova) are common for their clean, modern feel. Monospaced typefaces, often used in captions or for emphasis, add a slightly digital, “hacked” aesthetic that resonates with meme culture and internet-native visuals.

'''5. Outlined Text for Maximum Legibility'''<br />
Because TikTok videos often have busy backgrounds, outlined text is a practical necessity. White text with a black outline (or vice versa) ensures readability no matter what’s happening in the video, a trick borrowed from subtitles, gaming UI, and meme typography.

'''6. Text Motion'''<br />
Most of the transitions are a word-by-word replacement of the text. Resembling GIF animation, the text changes almost frame-by-frame in high-speed, following the AI reading rhythm. The (most of the time) fixed positioning is meant to allow the viewer to follow the text. However, both the fast replacement and the fact that there is no trace left of the text, creates a more skimming than reading effect.

'''What This Typography Say About TikTok’s Visual Culture'''<br />
TikTok typography is designed for speed, with text meant to be understood in milliseconds before viewers scroll away. Traditional rules of composition are often set aside in favor of clarity and immediate impact.

Bold, simple type draws on meme culture, emphasizing humour, emotion, and reaction-driven communication. Overall, TikTok typography prioritizes function over finesse, evolving text for viral, high-speed consumption.

<span id="an-analysis-of-typography-in-instagram-text-animations"></span>

==== An Analysis of Typography in Instagram Text-Animations ====

[[File:instagram_holke79_1.png|250px]] [[File:Instagram_kiel.d.m_2.png|250px]] [[File:instagram_luftformat1.png|250px]] [[File:instagram_mnkytype.png|250px]] [[File:instagram_nigina.dsgn_1.png|250px]] [[File:instagram_nigina.dsgn_3.png|250px]]

On Instagram you can find typographic experiments done by designers. These designs are closer to printed typography in spirit, but adding motion

Here are the key elements:<br />
'''1. All-Caps Dominance &amp; Sentence Case Variations'''<br />
Also on Instagram most of the text is set in all caps, presenting short, catchy sentences. When slightly longer texts are presented, they do appear in sentence case.

'''2. Black &amp; White (maybe One More Colour)'''<br />
The texts designed for Instagram take much more from printed publications aesthetics than the 90’s web features that are used on TikTok. There is much more diversity in the type choices and the colour combinations are softer. Black&amp;White and the grey tones dominate; some mix one simple colour as addition/ contrast to the grey.

'''3. Central Placement &amp; Clear Compositions'''<br />
Most text animations have their text centred, taking a large part of the format and maximizing the small mobile space. Animations where the text is being built-up use clear structures, either grid-based or a systematic composition.

'''4. Sans-Serif and Trendy Typefaces'''<br />
Sans-serif fonts are also common on Instagram. Next to Sans-Serif typefaces there is great use of more trending typefaces and a mixture of multiple types. This typographic approach puts more focus on enhancing legibility, making the text itself the main image and the background as supporting.

'''5. Harmonic Backgrounds for Maximum Legibility'''<br />
The designers’ work uses much more subtle backgrounds, that work harmonically with the presented text. The background is designed to work ''with'' the text, not overpower it.

'''6. Text Motion'''<br />
There is great diversity in the text motion. Some animations focus on changing the text’s visuality while the text itself remains fixed to its place. Other animations use the format to have text move in/out or around it. Immersive Text-style animations follow the reading rhythm, highlighting the words by changing the typeface style.<br />
There are also tools offering the reader interaction with the text itself for web-based content, but this might be less relevant for this specific prototype.

'''Captions'''<br />
Important part to add is the common use of text captions. There are many benefits for adding captions. As can be found in [https://www.usu.edu/accessibility/captions/benefits#:~:text=Adding%20captions%20to%20your%20videos,pay%20attention%20to%20the%20video. this] article by the Utah State University, captions assist in comprehension and memory and help users pay attention.

There are diverse visualisations of captions - word-by-word presentation of text, text in short lines, central single-word appearance and more. These captions do not follow intonation, rather seem to have definitions of space.<br />
Many of the platforms [https://www.google.com/search?q=instagram+add+captions&oq=instagram+add+captions&gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIGCAEQLhhA0gEINDgzMWowajSoAgCwAgE&sourceid=chrome&ie=UTF-8 supply with their own] captions' options, that will transcribe a video and let you style the text with their own pre-designs. When designing these captions, you can choose from a variety of typefaces, colours and adjust the sizes.

[[File:AmandaGormanInsta.png|250px]] [[File:InstaCaptions1.PNG|250px]] [[File:InstaText1.PNG|250px]]

''The prototypes''<br />
Building on this, we explored creating two immersive videos for the two platforms. The text was combined with motion, voice-over, and background imagery. This introduced new typographic challenges: composing layouts for small screens, maintaining clarity of typefaces, and aligning motion with reading rhythm.<br />
<br />
Our first sketch was for the typography only. The typeface '''Tiempos''' by [https://klim.co.nz/ Klim Type Foundry] was chosen for its legibility, deliberately contrasting with meme-style all-caps typography and outlined letterforms common on social media.

[IMAGE: SCREEN WITH TIEMPOS FONT]

We then moved to creating prototypes with motion. The prototypes tested our short text using different typefaces, layouts, playback speeds, and voice options. Clear differences emerged between platforms. '''TikTok typography''' is fast, high-contrast, and designed for skimming. '''Instagram typography''' is more structured and centered, supporting reading through rhythm and restrained motion.

For the ‘TikTok’ version, we chose to use background video recordings of the game ‘Minecraft’. This classic game is often used for storytelling videos on TikTok, unrelated to the story that is being told. For typeface we chose [https://fonts.google.com/specimen/PT+Mono PT Mono], a monospace, code-like typeface that will work well with the pixelated esthetics of the background images. The [https://bookaudiobook.net/one-hundred-years-of-solitude-audiobook/ audio] was available for download online, and we found it very suitable for our video with its deep, soothing voice.

For the ‘Instagram’ version, we chose a more subtle approach. For background, we wanted to check the effect of “hypnotizing” visuals, that will keep the reader engaged. We went for a black and white looping video of water ripples. For the text we used font [https://fonts.google.com/specimen/Merriweather?query=Merriweather Merriweather], a serif typeface that resembles a printed novel typeface. We animated the text with a ‘dropping’ effect, to create motion that follows the reading rhythm.

We eventually added a third prototype, using the opening text from the novel ‘War and Peace by Leo Tolstoy. For this video we based the text visualizations on the different characters: the narrator, Anna Pavlovna Scherer and Prince Vasili Kuragin. Each character got its own visual style, mixing [https://fonts.adobe.com/fonts/ff-meta serif] and [https://fonts.adobe.com/fonts/ff-meta sans serif] versions of Meta Pro and [https://fonts.adobe.com/fonts/mr-eaves-sans Mr. Eves Sans] with different colors.<br />
Also for this prototype we found an [https://archive.org/details/03Disc3Of4/01+-+Disc+1+of+4.mp3 audio] version online and used it for our test, since the reader was very distinguished in the way he characterized each person. That made the connection between written and spoken text much stronger.

[2X VIDEO PROTOTYPES]

Two prototypes were created using Adobe Premiere, one with Adobe After Effects. The greatest advantage of Adobe Premiere is its ability to transcribe captions based on voice. That is an automatic process that can be generated very fast for a long audio/ video part. The automatic generation, however, does not guarantee precision. It was necessary to review the text, adjust some words (mainly names) and edit manually the connection between background images and the text and its appearance in the layout, breaking some lines to adjust screen width with readability.<br />
<br />
Ideally, we would have an automated workflow that generates full video clips. This would allow content to be published quickly and at low cost. The technology behind this workflow should be free and open source, so anyone can use it to publish the content they choose. The video clips themselves are easily shareable and can be published both on one’s own platform and across existing platforms.

<span id="technology-1"></span>

=== TECHNOLOGY ===

<span id="text-to-speech"></span>
=== Text to Speech ===

An important aspect of immersive reading is voiceover. Tone, infliction, rhythm, and emphasis all play an important part in how the text is experienced by the reader. In an ideal scenario, there would be a human narrating the text, but in practice this is often too expensive and time-consuming. Therefore, we have done some experiments to determine the usability of various AI Text-to-Speech (TTS) technologies that could create voiceovers from text files.

<span id="ssml"></span>
==== SSML ====

After our initial exploration of various mainstream TTS products, the conclusion was that the voices sound quite convincing and pleasant, but the intonation is unnatural and does not properly emphasize the right words. We sought the solution to this problem in Speech Synthesis Markup Language (SSML). This is a way to include markings in de source text to indicate emphasis, pauses and other things that can improve the speech. Our idea was to use ChatGPT to add these markings to the source text, so that a TTS product can use this to improve its speech pattern.

<span id="comparing-tts"></span>
==== Comparing TTS ====

The next step was a more thorough auditing of various TTS services to determine their quality and their support for SSML.

'''Free option: Amazon Polly'''

https://aws.amazon.com/polly/

Amazon Polly provided unnatural sounding speech. There is a higher quality version available, but that could not be accessed from the Netherlands. It claims to support SSML, but its interpretation often sounds stunted.

'''Free option: Crikk'''

https://crikk.com/

Crikk works better out-of-the-box, but it does not support SSML so it can not be improved upon. In does support pauses, but nothing else.

'''Free option: Google TTS'''

https://cloud.google.com/text-to-speech

Google TTS works better than Amazon and supports SSML, but still the intonation remains very unnatural.

'''Paid option: Elevenlabs'''

https://elevenlabs.io/

Elevenlabs offers a trial of 10000 characters per month. It works much better than all the other options. It appears to do it’s own preprocessing to figure out the right intonation for each sentence. It doesn’t support SSML, but frankly it doesn’t need it.

Overall, there is more variation in quality between different TTS services than we expected, and the large players don’t necessarily seem to do best. However, looking at the pace of innovation in this area, we expect the quality to improve drastically in the coming years.

<span id="elective-immersive-reading"></span>