Difference between revisions of "Courses/Hybrid publishing resources"

From Publication Station
 
(31 intermediate revisions by the same user not shown)
Line 6: Line 6:


==Focus==
==Focus==
'''Transforming a source (manuscript) into multiple publishable outputs'''
'''Transforming a source (manuscript) into multiple publishable outputs, within a hybrid publishing workflow'''
 
'''In a Hybrid Publishing Workflow'''  


==Hybrid Publishing Workflow==
==Hybrid Publishing Workflow==
Line 21: Line 19:


[https://raw.githubusercontent.com/DigitalPublishingToolkit/Hybrid-Publishing-Toolkit-for-the-Arts/master/trailer-From_Print_to_Ebooks:_a_Hybrid_Publishing_Toolkit_for_the_Arts.gif EPUB trailer]
[https://raw.githubusercontent.com/DigitalPublishingToolkit/Hybrid-Publishing-Toolkit-for-the-Arts/master/trailer-From_Print_to_Ebooks:_a_Hybrid_Publishing_Toolkit_for_the_Arts.gif EPUB trailer]




Line 29: Line 26:
== Structure ==
== Structure ==
By putting the '''content into an explicit structure''', which can withstand those transformations.
By putting the '''content into an explicit structure''', which can withstand those transformations.
So what is a title  (heading) in a format, remains a title when translated to a different format.


==  How to achieve explicit structure?  ==
==  How to achieve explicit structure?  ==
Line 76: Line 75:
</pre>
</pre>


== pandoc ==
== Pandoc ==
[http://johnmacfarlane.net/pandoc '''pandoc''']: software for converting between markups with
[http://johnmacfarlane.net/pandoc '''pandoc''']: software for converting between markups with


http://johnmacfarlane.net/pandoc/diagram.png
[[File:pandoc_diagram.png]]
 


== ****************** ==


== markup a text in MS Word/Libre Office/Open Office ==
: styles; Test on WDKA


== mark a text in MS Word/Libre Office ==
create a '''.docx''' file and mark it up with a few of the following paragraph and characters styles:
create a '''.docx''' file and mark it up with a few of the following paragraph and characters styles:
* headings
* headings
Line 102: Line 102:
:shell
:shell


:Pandoc command
<pre>pandoc -f docx -t html5 -s input.docx -o output.html</pre>
 
== convert once more, now to EPUB ==
Can you instead of converting the .docx into HTML convert into an EPUB?
 
: clues
 
== Markdown ==
Markdown as the '''source file'''.
 
: link to Markdown syntax
 
Any plain text editor such as [http://www.sublimetext.com/ Sublime Text] or [https://wiki.gnome.org/Apps/Gedit Gedit] can be used to edit Markdown files. There are also WYSIWYG Markdown editors such as [http://macdown.uranusjr.com/ MacDown].
 
== Why Markdown ? ==
* simple
* only '''one way''' to create a structure 
* compatible with HTML, but easier to read and write
* allows the inclusion of HTML tags. E.g.  <pre><!-- an html comment -->  <span id="mrspan">a div</span></pre>
== convert once more, now to Markdown ==
 
: pandoc argument
 
 
== **** ==
 
== stream line: makefile ==
This same approach of converting between markups using Pandoc can be automated through scripts that execute the commands we have been writing in the terminal.
 
Ands since computers are good at doing repetitive and boring tasks, we can take advantage from it.
 
We can make them perform the same task on multiple source files, and make them produce multiple outputs (E.G. EPUB, ICML, HTML).
 
== makefile ==
My proposal involves using a Makefile to automate conversions between markup-languages
 
Makefiles are usually used to in the free-software community to compile source code into running applications.
However they are not more than a notebook of commands to process stuff and generate other stuff from it: e.g. create executables programs out of source code files. There is nothing stopping us from using makefiles to automate conversions between markup-languages.
 
Read more about Makefiles in the context of hybrid publishing:
* the blog post [http://networkcultures.org/digitalpublishing/2014/10/01/make-book/ Make Book] by Michael Murtaugh
* the [http://lwn.net/Articles/589196/ report] on the Don Marti's presentation at SCALE
* [http://www.gnu.org/software/make/ GNU Make official site ]
 
 
== overview ==
The conversion from a manuscript to EPUB it is a step by step process, that consists of:
 
# '''.docx - Manuscript''' - .docx - editing the manuscript according to the INC style guide
# '''Markdown - Source''' - converting the manuscript files to Markdown files.
# '''ICML files for inDesign''' - converting the individual Markdown source into ICML files that can be imported into inDesign
# '''EPUB Output''' -converting the compound Markdown source file to an EPUB
 
Note: The section &quot;6.3. Do-it-yourself EPUB using Pandoc&quot; form [[From%20Print%20to%20Ebooks|http://networkcultures.org/blog/publication/from-print-to-ebooks-a-hybrid-publishing-toolkit-for-the-arts/]] is an important complement to this tutorial and set of tools
 
[[Image:http://networkcultures.org/wp-content/uploads/2015/02/workflow.png]]
 
 
 
== (install Make) ==
* Download http://rudix.org/download/2014/10.8/make-3.82-4.pkg
* Using the finder go to Downloads and begin installing that package
 
== clone Hybrid Publishing Resources  ==
* Download the [https://gitlab.com/DigitalPublishingToolkit/Hybrid-Publishing-Resources/tree/wdka code repository] into your computer. Download link: https://gitlab.com/DigitalPublishingToolkit/Hybrid-Publishing-Resources/repository/archive.zip?ref=wdka
* Move the downloaded .zip into a dedicated folder (inside DATASTORAGE in WdKA computers).
* Unzip
* In terminal change directory <code>cd </code> to the unzip folder
 
 
== download content ===
* Download http://stuff2233.club/~iceking/content.zip
* Unzip
 
content.zip contains a couple of .docx files, which you will use as your content.  Any other structured (with Styles).docx files could be used.
 
There is also a .ttf font file and an image.
 
 
 
----
 
==start cooking==
 
 
== 0. Folder structure ==
 
To create folder structure, which will be understood by the scripts from this repository, you need to run:
 
<code>./scripts/create_folders.sh</code>
 
This script will generate the following folder structure, which you'll use to store the files essentially for the creation of the EPUB. <code>├── docx ├── epub ├── icml ├── md │  └── imgs └── scripts</code>
 
 
 
 
== 1. Manuscript: .docx ==
 
This is a preparatory stage. Yet it is highly important for the series of conversions that will lead to the different publication's outputs.
 
You'll be editing the text document handed by the author - the manuscript -, according to [[INC styleguide]]: section &quot;Document Formatting/Layout&quot;, subsection &quot;General&quot;.
 
Do pay special attention to this part of the [[INC styleguide]]. Following it rigorously will ensure that your manuscripts will convert flawlessly to markdown (the next step).
 
== 2. Source: converting the manuscript files to Markdown files ==
 
Run: <code>make markdowns</code> to start this stage.
 
This command will '''convert all the <code>.docx</code> files inside the <code>docx/</code>folder into corresponding markdown files inside <code>md/ folder</code>'''.
 
The resulting '''markdown files are meant as source files''', from which all of the publication's outputs will be generated.
 
The reason for using markdown as a the ''source format'' comes from easiness by which they can be read and written, they high compatibility with other markup languages (HTML, ICML) and the corresponding outputs (EPUB/Website, inDesign Project)
 
=== 2.1 Images in Markdown ===
 
At this stage you should insert the images in to the essay, now in markdown format.
 
First of all, you need to '''save all the images included in the essays in the folder''' <code>md/imgs/</code> Then '''place the images on the markdown files''': <code>![My image caption](imgs/myImage.jpg)</code>
 
Don't forget to '''include captions''', if the image has them. Markdown captions will become visible, and associated to the image in the EPUB output
 
=== 2.2 Metadata in Markdown ===
 
Each of the resulting markdown files contain a '''metadata header'''. You can fill those fields with the corresponding information.
 
Note on metadata: You can change the metadata fields and values that are added to each article, by editing the template file used to generate the markdown files: <code>essay.md.template</code>
 
'''Masterclass:''' In <code>md/1_1-Hart_Keith.md</code> complete the metadata, and insert images in their correct location.
 
== 3 ICML files for inDesign ==
 
Run: <code>make icmls</code> to start this stage.
 
The individual Markdown source files, stored in <code>md/</code>, can be converted into ICML files which can be imported into inDesign.
 
ICML files are useful, since they ensure that the structural information of the Markdown source files is also present in the inDesign projects, which use them.
 
Another point in favor of ICML is the possibility of updating the content and structure of the inDesign projects, by updating the Markdown source files, converting the once more to ICML. For this to happens is however necessary that inDesign remains linked to its source, and that the designers works with paragraph and chapter styles, instead of directly into the text.
 
'''Note: ''malformatted links in the markdown files'' will create problems when imported into inDesign.'''
 
= 4. '''Outputs''' =
 
== (Assembling the Markdown source files into book.md) ==
 
This step is essential to the creation of the EPUB. However, '''you don't have to perform this step, as the makefile does it for you'''.
 
It generates a single Markdown file <code>book.md</code> and save it it inside the <code>md/</code> folder. <code>book.md</code> is comprised of the content from all the individual Markdown source files inside the <code>md/</code> folder, in alphabetical order (00 to ZZ).
 
The <code>book.md</code> file will, in step 4.1 give origin to the
 
If you need for any reason to generate this single Markdown file <code>book.md</code>, run: <code>make book.md</code> .
 
== 4.1 EPUB Output ==
 
<code>book.epub</code>
 
Run: <code>make book.epub</code> to start this stage.
 
=== Essential files: ===
 
To produce an EPUB a few resources (files) are needed, namely the cover image, metadata, stylesheet, and fonts.
 
This files will strongly influence the EPUB's outcome, and consequently should be edited for each publication or series of publication
 
* <code>epub/metadata.xml</code> - EPUB metadata
* <code>epub/styles.epub.css</code> - EPUB css style-sheet '''MUST BE EDITED'''
* <code>epub/cover.jpg</code> - EPUB cover '''MUST BE CHANGED'''
 
=== Optional fonts: ===
 
* lib/ - folder for storing custom fonts, that will be used in the EPUB
 
'''Note on the use of custom fonts''': If you choose to use fonts, make sure to change the makefile to include the use of fonts in the makefile epub rule, such as in the rule bellow, where <code>--epub-embed-font=lib/UbuntuMono-B.ttf \</code> was added to allow for the use of the Ubuntu Mono font. Also include the font on th EPUB style-sheet with <code>@font-face</code> rule
 
== 4.2 EPUB check ==
 
The health of the created EPUB can be checked with http://validator.idpf.org/
 
== 4.3 EPUB edit ==
 
Once it has been created the EPUB can be change manually by using Calibre's recent tool [http://manual.calibre-ebook.com/edit.html Edit Book]
 
== the makefile ==
 
 
 
== ***** ==
 
== IMCL ==
: to be edited
 
== Recipe Ingredients ==
* '''empty folder''' - to create the recipe
* '''content''' in a markup language: HTML (or wiki syntax)
* '''terminal''' - a text-based interface to your file system
* [http://johnmacfarlane.net/pandoc/installing.html '''pandoc'''] - markup converter software
* '''plain-text editor''': [http://www.sublimetext.com/ Sublime Text] or [https://wiki.gnome.org/Apps/Gedit Gedit]
* your favorite text layout software
 
== Testing the Recipe ==
 
== Testing the Recipe step#1 ==
 
In on empty folder in your computer save 1  BS article in '''HTML'''.
 
== Testing the Recipe step#2 ==
'''Convert that HTML file into an ICML file.'''
 
<pre>pandoc -f html -t icml -s input.html -o output.icml</pre>


'''pandoc''' - program dedicate to the conversion between different markups.
'''pandoc''' - program dedicate to the conversion between different markups.
Line 325: Line 114:
'''-o''' - option for file output;
'''-o''' - option for file output;


'''input.html''' - html input filename - you need to replace it by its actual name
'''input.docx''' - docx input filename - you need to replace it by its actual name
 
== Testing the Recipe step#3 ==
Open inDesign and ...
 
'''Place your output.icml in a inDesign project.'''
 
The Place function is in: File > Place (shortcut: Apple+D)
 
== Testing the Recipe step#4 ==
'''Style the content using paragraph and character styles. '''
 
Paragraph/Characters styles are in: Type > Paragraph/Character Styles
 
 
== Testing the Recipe step#5 ==
'''Content updates'''
 
Using Sublime Text, edit the source html file adding something or extracting something to it.
 
Repeat the whole recipe, so that you see the content being updated in inDesign.
 
== Recipe Warning ==
'''Content should remain untouched until the last moment'''
 
If you change the content of the articles inDesign, the link between the inDesign content and its source ICML file will be lost, and it no longer be able to be updated.
 
This "disconnection" should only be the last step to be done on your design workflow.
 
== Recipe Tips ==
'''Loading paragraph styles'''
 
==Recipe Questions==
'''Can CSS rules be imported and used as inDesign Paragraph/Character Styles ?'''
 
'''Can inDesign styles be exported as CSS style-sheets ?'''
 
'''+ ... ?'''
 
 


== convert once more, now toward Markdown ==
<code>-t markdown</code> Option: toward Markdown
<code>--atx-headeres</code> Option: use nice Markdown headers


== Why Markdown ? ==
* simple
* only '''one way''' to create a structure 
* compatible with HTML, but easier to read and write
* allows the inclusion of HTML tags.
* any plain text editor such as [http://www.sublimetext.com/ Sublime Text] or [https://wiki.gnome.org/Apps/Gedit Gedit] can edit Markdown files.
* there are WYSIWYG Markdown editors such as [http://macdown.uranusjr.com/ MacDown].


== ******* ==






{{:Hybrid_publishing_resources}}


==software used==
==software used==

Latest revision as of 09:33, 26 May 2016

<slidy theme="a" />

publicationstation.wdka.hro.nl/wiki

Hybrid Publishing

Focus

Transforming a source (manuscript) into multiple publishable outputs, within a hybrid publishing workflow

Hybrid Publishing Workflow

  1. various outcomes from one workflow
  2. constant connection between source-content and outcome
  3. avoiding repeating tasks

Hybrid Publishing Workflow

Hybrid-workflow.png

http://networkcultures.org/blog/publication/from-print-to-ebooks-a-hybrid-publishing-toolkit-for-the-arts/

EPUB trailer


How?

How can we achieve a publishing workflow where we can go from the source (a word document, a wiki page, a webpage) to several outputs (an EPUB, a website, an animated gif, an inDesign project)?

Structure

By putting the content into an explicit structure, which can withstand those transformations.

So what is a title (heading) in a format, remains a title when translated to a different format.

How to achieve explicit structure?

An explicit structure is achieved by marking the text with structural information.

marking the text = markup

Markup languages

HTML:

<h1>Revenge of the Text</h1>
 <p>There is a room in the <strong>Musée d’Orsay</strong> that I call the <em>room of possibilities</em>.</p>
 <p>That room contains:</p>
 <ul>
  <li>a snow flake</li>
  <li>the end of a cloud</li>
  <li>a bit of nothing</li>
 </ul>

Wiki markup:


= Revenge of the Text =

There is a room in the '''Musée d’Orsay''' that I call the ''room of possibilities''.

That room contains:

* a snow flake
* the end of a cloud
* a bit of nothing

Markdown:

# Revenge of the Text

There is a room in the **Musée d’Orsay** that I call the *room of possibilities*.

That room contains:

* a snow flake
* the end of a cloud
* a bit of nothing

Pandoc

pandoc: software for converting between markups with

Pandoc diagram.png


******************

mark a text in MS Word/Libre Office

create a .docx file and mark it up with a few of the following paragraph and characters styles:

  • headings
  • body text
  • block quotes: indented blocks of text
  • footnotes
  • hyperlinks
  • Preformatted text
  • Bold
  • Italics

Save in .docx

convert

Convert your docx file into a an HTML file using Pandoc.

shell
pandoc -f docx -t html5 -s input.docx -o output.html

pandoc - program dedicate to the conversion between different markups.

-f - option standing for “from”, is followed by the input format;

-t - option standing for “to”, is followed by the output format;

-s - option standing for “standalone”, produces output with an appropriate header and footer;

-o - option for file output;

input.docx - docx input filename - you need to replace it by its actual name

convert once more, now toward Markdown

-t markdown Option: toward Markdown

--atx-headeres Option: use nice Markdown headers

Why Markdown ?

  • simple
  • only one way to create a structure
  • compatible with HTML, but easier to read and write
  • allows the inclusion of HTML tags.
  • any plain text editor such as Sublime Text or Gedit can edit Markdown files.
  • there are WYSIWYG Markdown editors such as MacDown.

*******

streamlining conversions

This same approach for converting between markups using Pandoc can be automated.

Since computers are good at doing repetitive and boring tasks, we can take advantage from it.

We can store commands in scripts and have them perform the same task on multiple source files, producing multiple outputs (E.G. EPUB, ICML, HTML).

Overview

In this process, from several .docx files we'll try to create two outputs:

  • an ICML file for inDesign
  • an EPUB

And we'll also try to keep a connection between the source and its outcomes.


Makefile

My proposal involves using a Makefile to automate the conversions between markups.

They are like recipe book, with instructions of how to process stuff and generate other stuff from it.

Read more about Makefiles in the context of hybrid publishing:


Recipe Preparation

Download Hybrid Publishing Resources


Start Cooking

0. Folder structure

To create folder structure, which will be understood by the scripts from this repository, you need to run:

make folders

This script will generate the following folder structure, which you'll use to store the files essentially for the creation of the EPUB.

├── docx 
├── epub 
├── icml 
├── md 
│   └── imgs 
└── scripts

1. Manuscript: .docx

This is a preparatory stage. Yet it is important for the series of conversions that will lead to the different publication's outputs.

You need to edit the manuscript, handed by the author, using paragraph styles in MS Word/Libre Office/Open Office


2. Source: converting the manuscript files to Markdown files

Run: make markdowns to start this stage.

This command will convert all the .docx files inside the docx/folder into corresponding markdown files inside md/ folder.

The resulting markdown files are the source files, from which all of the publication's outputs will be generated.


2.1 Images in Markdown

At this stage you should insert the images onto the texts, now in markdown format.

First of all, you need to save all the images included in the essays in the folder md/imgs/ Then place the images on the markdown document: ![My image caption](imgs/myImage.jpg)

Don't forget to include captions, if the image has them. Markdown captions will become visible, and associated to the image in the EPUB output.

3. make icmls - ICML files for inDesign

Run: make icmls to start this stage.

The individual Markdown source files, stored in md/, can be converted into ICML files which can be imported into inDesign.

Read more on ICML files and inDesign in #working_with_ICML_files section


4. make book.epub

4.0 make book.md

You don't have to perform this step, the makefile does it for you.

It generates a single Markdown file book.md and save it it inside the md/ folder.

book.md consists of the content from all the individual Markdow files inside the md/ folder, in alphabetical order (00 to ZZ).

If you want to change the order the publication, you can change the filenames in order to do so.

Markdown filenames should NOT contain spaces or dots, except for the extension .md


4.1 make book.epub (EPUB)

Produce the epub file: make book.epub


4.1.1 Essential files for EPUB

To produce an EPUB a few files are needed, namely:

  • cover image epub/cover.jpg
  • metadata epub/metadata.xml
  • CSS stylesheet epub/styles.epub.css


These files will strongly influence the EPUB's outcome, and consequently should be edited for each publication.


4.1.2 Fonts (optional) for EPUB

lib/ is folder for storing custom fonts, that will be used in the EPUB

If you choose to use fonts, make sure to change the makefile to include the use of fonts in the makefile epub rule, such as in the rule bellow, where --epub-embed-font=lib/UbuntuMono-B.ttf \ is added to allow for the use of the Ubuntu Mono font. Also include the font on the EPUB style-sheet with @font-face rule

4.1.3 EPUB check

The health of the created EPUB can be checked with http://validator.idpf.org/


4.1.5 make book.epub : An iterative process

This work is in many way as cyclical process, where you work in circles: changing the source files (markdown files, the EPUB's stylesheet, the EPUB's metadata) and seeing the results of those changes in Calibre (or other ebook-viwerer)



ICML files

ICML files are useful, since they ensure that the structural information of the Markdown source files is also present in the inDesign projects, which use them.

With ICML it is also possible to update the content and structure of the inDesign projects, by changing the Markdown source files, converting the once more to ICML. For this to happens is however necessary that inDesign remains linked to its source, and that the designers works with paragraph and chapter styles, instead of directly into the text.

Note: malformatted links in the markdown files will create problems when imported into inDesign.


Place ICMLs

Open inDesign and ...

Place your output.icml in a inDesign project.

The Place function is in: File > Place (shortcut: Apple+D)

Design with Styles

Style the content using paragraph and character styles.

Paragraph/Characters styles are in: Type > Paragraph/Character Styles

ICML Content updates

Using Sublime Text, edit one ICML file adding some text to it.

See the content being updated in inDesign.


Hands off content

In in inDesgin content shall remain untouched until the last moment

If you change the content of the articles inDesign, the link between the inDesign content and its source ICML file will be lost, and it no longer be able to be updated.

This "disconnection" should only be the last step to be done on your design workflow.

software used