[This post is purely for people interested in technical printing issues: probably other authors who use print-on-demand services, rather than readers. Although based around using Lulu, much of this would also apply if using another print-on-demand (PoD) service such as CreateSpace or Lightning Source. You have been warned. Venture no further into this nitty gritty morass unless you are ready to wrestle with the tangly blood-vines.]

There's a reason some people are put off DIY.

I had finished 2000 Tunes. I printed it at home - perfect. I ordered proof copies from Lulu - perfect. I enabled global distribution - no errors. All the checks were done and the machinery gathered steam. My print book appeared on distribution sites such as Amazon. I was pleased.

But hubris is often dashed by the gods.

Out of the blue, a few weeks after I thought it was done and dusted, I received this email from Lulu:

During the review of your book "2000 Tunes" (Project ID: 16726697) for distribution we received an error notification from our distribution partner:
- There are images and/or text that are not clear enough to print. All images must have a resolution of at least 300dpi; any images that do not meet this requirement must be removed or replaced.
- There are overlapping text layers or cut off text in your manuscript.  Text must be legible and may not overlap or be cut off.
- The margins of your manuscript are not large enough.   Please revise your manuscript’s margins so that all text and images are at least .5” away from the edge of the page, or format your document for full bleed printing:http://connect.lulu.com/t5/Interior-Formatting/How-do-I-make-my-content-stretch-all-the-way-to-the-edge-of-the/ta-p/33675
- The interior does not have sufficient gutter/outside margins for printing on PDF pages: 5,27,45,63,93,111,125,145,173,193,205,237,285,305,351,359,469,470.Blurry images and/or text on pages:5,17,27,45,63,73,83,111,125,145,193, 205,305,313,351,359,375,385,407.;;Cut off text on pages:5,17,27,45,63,73,83,93,111,125,145,173,193, 205,237,259,285,305,313,351,359,375,385,391,407.
Please revise your cover and/or manuscript file(s) to correct this error.

Your project will remain available on Lulu.com but will not be listed on retail sites until the necessary revisions have been made.  For a full list of the mandatory distribution requirements, please see the following page:
What did all that mean? I'd used high resolution images, I'd set it up so it printed well, I'd ordered copies with no problem... After days of formatting and approving various proof copies (a significant cost in time and money) I was filled with visions of orders being cancelled and my print book disappearing from suppliers such as Amazon. I contacted Lulu to find out more, pointing out that the things listed as errors were incorrectly identified. 

1. The margins were the same (correct) throughout the book. 
2. The full-page images (which were crystal clear - I had the Lulu-printed book in front of me) had built-in black borders which could be cropped. It looked absolutely perfect, and had in every copy ordered. I didn't want them surrounded by a white border like the main book text because then they'd look weird, as well as making the images smaller; also, since they were scrapbook cuttings from another fictional book with their own page numbering system it would look stupid if they didn’t fill page, and then had two page numbers above each other.
3. I could understand why there are general guidelines but they don't always apply e.g. I lowered the DPI slightly because otherwise the book file was too large, but as with all careful compression the end result was indistinguishable from the uncompressed file, whilst reducing the overall project from 100MB to 10MB. 

I worked with Lulu to understand what was going on and how it could be over-ridden or fixed. It was tricky to unravel - Lulu had launched a new process the day before (14th July) called PDF Normalization which aimed to help to identify concerns early on, prior to being sent to print, so that users were aware of things that might cause conflicts during production. I learnt that Lulu's printers had always manually added 1/8" of additional white space to any user's books that do not include bleed (I'll talk about bleed later), but the new system was that Lulu would now be adding the 1/8" white borders rather than the printer, in order to allow users that have artwork at the edges of their pages the opportunity to make changes prior to receiving defective artwork. Mainly due to the fact that if you leave the artwork on the edge of a page without properly adding bleed, it will inevitably be printed cropped into or with thin white edges as a result. The aim was to allow users to correct these problems before they occur. At first I was told I didn't need to do anything, since I could continue as I always had.


But then on further investigation we discovered that the new process was a red herring which had just started at the same time: the messages were very similar, but mine was genuinely from Lulu's distributor. Basically, when you enable global distribution of a book on Lulu there is a 6-8 week process to see if the book fulfils criteria set by distributors. The distributors are given access to the files that will be used for print so that they can make sure they meet their specific requirements. So this was much more serious. It meant no customers could order copies of the book until it had been approved by their distribution partners. Sites like Barnes and Noble and Amazon are very particular with the specifications of what they offer on their sites (regardless of whether they always affect the output), and Lulu had no say in those criteria.

It's worth noting that the book had previously been accepted with the formatting I'd used. So there was no way I could have known that later on the distributor would change their mind. As Lulu said (which includes some other useful information about how distribution works):
"It looks like our retail partners accepted your previous version of "2000 Tunes" project #16726697 which you approved the proof of on 10 June 2015. This is the version available for sale through retailers currently and should remain available for sale. If a customer places an order through a retail site such as Amazon currently this is the version that they would receive.
Lulu submits eligible print books to two retail channels, directly to Amazon and to Lightning Source / Ingram. This helps ensure that books are available for sale across the most retail sites. Amazon sent us the notification which we passed along to you.
Amazon's system also often automatically rejects images or text that are cut off even if it is intentional, such as the clippings at beginning of each chapter. We can sometimes push back on Amazon by requesting that they re-review the matter which generally takes them an additional few weeks. However, the final decision is up to them on whether or not they accept a book as-is or require changes."
So I faced trying to understand how to translate vague error messages into concrete steps to take in Word to alter the document whilst also recording them all for future reference.

I still didn’t know exactly what the impact of this error message would be: would existing book orders from distributors be cancelled, or just paused and fulfilled later? A bit of a nightmare, because I was due to go away for a week-long residential course I needed to prepare for.

Nothing else for it. No point tearing my (limited) hair out, wailing and gnashing my teeth any longer, because as well as being an art, and a craft, writing is a business, and that requires being professional and just doing what needs to be done. Sometimes that means getting your hands dirty. It was time to get grubby.

Margins/Bleed Error
I decided to tackle bleed and page size first.

The kind of diagram I had to work with

The error message said:
“The margins of your manuscript are not large enough. Please revise your manuscript’s margins so that all text and images are at least .5” away from the edge of the page, or format your document for full bleed printing”
“The interior does not have sufficient gutter/outside margins for printing on PDF pages”

These two errors meant more or less the same thing, so I hoped that fixing one would fix both. I asked for some clarification and was told:
"Note that the required amount of bleed for any interior file that has artwork going to the very edge of a page is 1/8". So in a 5.83 x 8.27" book, you want to make sure every page is instead  6.08 x 8.52". I will attach an image called "Bleed_Margin_Guide" for you to reference. In the image, the outside red area represents the 1/8" bleed area that is guaranteed to be cut off at any point, and the inner blue area is the 1/4" "danger" margin where anything could possibly be cut into due to common variance during production. Keep all text and images inside of these margins unless you intend for them to "bleed" off. [...]
If a file has art going to the edge, it needs 1/8" of bleed around every page of the interior file (i.e. 8.5x11"---->8.75x11.25"). The margins are completely separate from the bleed though, and are the area between the edge of a page and the next closest object. Then the gutter is an additional area added to the margin where necessary (But I don't think is required).
So at the very least, your book requires a .5" margin on each page. The rest is completely dependent on the design of your interior.
Because of PDF Optimization, all books that do not currently have the additional 1/8" necessary space, automatically have it added to their file when uploaded to a project on Lulu.com. It is very important to understand that if the file has art going to the very edge, that all we are doing is adding an additional 1/8" of white border around each page per our printers file-size requirements. This is not a new process, and our printer has manually done this step as needed in the past, but if you have art/text/design/color that goes to the edge of a page that isn't sized correctly for bleed, it will more than likely come back cropped into or with thin white edges."
I'd originally set the page size as A5 to match what Lulu would print - it seemed like common sense. But now I had to make the “page” in Word bigger than A5, using a custom size, by working out the A5 dimensions and adding ¼ inch (2 x 1/8 inch) to height and width. However, I realised that would reflow the text into the larger space of this bigger page (since the margins would stay the same), so then I’d need to expand the margins by the same dimensions and reflow the text back into the space in the middle.

An immediate complication arose. The Lulu guide on adding bleed said to add .635 cm to the page width and height. A5 = 14.8 x 21cm, so for A5 with full page images, according to Lulu’s guide this should be: 15.435 x 21.635. But the same Lulu page says: “If the page size is larger than the sizes on the right hand side of the table below, the PDF will be resized without bleed”. For A5 on the table, the right column “PDF Page Size with Bleed” gives the dimensions 15.44 x 21.62 cm. So if you add the height Lulu recommends (to make 21.635cm), the page is then too big according to this (21.62cm) and will be resized! Which makes it impossible to add .635cm to the height as requested. Were some of the figures in that Lulu table incorrect, such as the A5 one? Should I resize to 21.62cm, but possibly then not have added enough bleed and risk getting rejected all over again; or should I resize to 21.635cm, but then possibly Lulu will resize it without bleed, and again lead to the same errors and rejection? Oh joy. 

So this was my procedure.
  • A5 = 14.8 x 21cm
  • For full page bleed I added .635 cm to width and height.
  • So for A5 with full page images, custom paper size = 15.435 x 21.635
  • [Word will round it up/down, so maybe choose 15.44 x 21.64 cm]
  • After doing this the text expands and page numbers reduce as Word thinks it has a bigger page to play with. So I had to expand the margins to counteract that. Mine are 1.6 cm all the way round, so now each one needed to be 0.318cm larger, = 1.9cm more or less. This takes the page numbers up again.
  • Of course, every time you change settings like these, you will need to go through every page of your book checking for widows and orphans, page breaks that no longer apply, chapters beginning on odd-numbered pages and so on (do it with formatting marks showing). This took multiple passes, one check in each. I find it easiest to use the view with two pages side by side.
  • Then at the end I could update the page numbers in the table of contents (ToC) and re-centre the table (the page/margin changes had shifted it).
Hopefully that would fix it. My only worry was this note in an email from Lulu:
"Regarding the size of your files. If you are creating an A5 book, definitely work with an A5 layout and don't adjust the size of the file if you don't need to. If you submit a different sized file to our uploader, it will resize the document and this opens the door for other issues.The best way to proceed depends entirely on the design of your interior. If you have artwork that goes to the very edge of the interior, you will need the additional 1/8" bleed. If you do not have any artwork that you are worried about being too close to the edge, just let Lulu add the bleed manually for you."
This suggests I should have just left it as A5, or resize back to A5. However it was A5 originally, and that is what triggered the errors, stopping distribution, and led to me being told to add extra space/bleed beyond the normal paper size… So would my fix create problems? This is what I mean by the confusion. [I have yet to receive clarification on this from Lulu.]

Image DPI (PPI)
"There are images and/or text that are not clear enough to print. All images must have a resolution of at least 300dpi; any images that do not meet this requirement must be removed or replaced."

This was the second thing to look at. It taught me a few things that could be useful to other authors using images in their work, since there are a number of traps here.

Microsoft Word turns it into a rather high hurdle because even if you set images at the correct DPI, MS Word will lower the DPI automatically in at least three occasions: firstly when you save the document (unless you have changed one of the buried default options); secondly if you do anything to modify or resize the image in Word; thirdly if you export to PDF (which you need to do to embed fonts for uploading to a PoD service). There is lots of information about this on Createspace forums here, here, here and here.

Make sure fonts are embedded in the settings in Word

It's not easy to check an image's DPI in Word, since it will be affected by resolution AND how big the image is to be printed (i.e. the paper size). You should be able to just right click on an image in Word and find out DPI based on paper size and resolution, but you can't. (And Word should keep images uncompressed or warn you if it is going to compress them - but it doesn't). Remind me why we use Microsoft word again?

So I needed to change the images to the original hi-res ones again, hopefully with a high enough DPI.

Do not compress images in my file, Word!
  • Remove the compression option first File>Options>Advanced. You can do it either just for the one document (since it normally doesn't cause problems, only in books with images required for printing), or make it the options for All New Documents. Warning! If you set it for all new documents, you have to go in again and also set it for the current document. Otherwise Word assumes "all documents except this one".
  • I then right-clicked on each image, selected Change Picture (which works as long as the file is in .docx format, not .doc), and replaced the versions Word had secretly compressed with the bigger originals. I knew I'd have to use a different option to get an uploadable PDF, which I'll cover later.
Overlapping/hidden text
“There are overlapping text layers or cut off text in your manuscript.  Text must be legible and may not overlap or be cut off.”

I wasn't sure what this error meant. The only thing I could think of was that behind the images were chapter titles – not visible, but used to generate the automated table of contents. It was nothing the user would ever see but the distributor made the (incorrect) assumption that obscured text was an error, and there was no way to argue with them. I had to change it. I did this last because it involved destroying document navigation, making replacing images much harder...

Interior navigation in my novel in Word - lost
when I had to nuke the hidden headings
  • So I needed to lose the hidden headings and the automated ToC. First I pasted the ToC into notepad then back in to Word, so I had a plain text set of contents. I now have to manually check page numbers after all changes in future. 
  • Time to delete hidden chapter headings and associated bookmarks. I dragged each image out of the way, deleted the text/heading, watched my lovely left navigation disappear. It was like seeing a friend leave forever on a big ship - it had been such a savior in a 138,000 word novel. It shows why you should do images and presentational formatting as the last thing of all when you write a book, after every word is finished and final!
  • [To be tidy I also used Alt-I-K to delete bookmarks (ToC entries) one by one - Word won't let you select more than one at a time.]
  • Tip: Don't right-click on an item in the left-hand navigation menu and choose Delete - instead of deleting the heading, it will delete the whole section/chapter!
Preparing the document for uploading to Lulu
There's a checklist I use for each edit now.
  • Check the ToC page numbers for each chapter are correct. The easiest way = Print Screen to capture an image of the ToC, paste it into a graphics package. I can then alt-tab between it and the Word doc, not have to go back to the start of the Word doc every time to check it has the correct page number for the chapter; instead I can just move through doc sequentially. I use F5 (Go To), type each page number from the screenshot, Enter, check it's correct and the chapter starts there, alt tab to image, repeat.
  • Then I have to prepare the PDF using special software - Word's "Save as" PDF option compresses the images below 300 DPI without warning (220 DPI is the maximum it lets you use in Word). This would reset all your good work on the images. So I installed doPDF. Instead of saving the document as a PDF I "print" it, and select doPDF as if it was a printer. Printer Properties = the settings, so I can set the DPI as 300 etc. Make sure "Page size" matches the the Word document Paper size (and note that doPDF only offers custom sizes in mm, not cm, so don't forget to multiply the cm by 10). For my book the dimensions are 154.4 mm by 216.4 mm. Unfortunately doPDF doesn't remember settings, so I have to do this every time! 
  • When I click Print a popup allows me to set a save location, quality settings etc. I went for High quality (taking no risks), and ticked Embed fonts (since my novel does use different fonts for a few things such as hand-written letters, text messages etc). You could use lower quality if emailing a PDF to someone. In my case the filesize differences were:
    High 34mb
    Medium 9mb
    Smallest 3mb
At the end
So I spent days understanding all this and made many changes to my document. Unfortunately there is no fast-track way to run it through the distributor's checker to confirm that the issues are fixed and it is acceptable to them. You have to wait a few weeks for that confirmation. 

In future I will avoid full-screen images, it is more trouble than it is worth! Their flawless appearance in proof copies does not mean the distributor will accept them. For 2000 Tunes they're so integral to a twist in the plot that I can't separate them out, but never again!

The final bump was that after all that work I received a proof copy with an overly-cropped cover. I naturally assumed this was caused by my alterations to the interior due to the timing, flollowing all my changes. I then spent another day moving cover elements around, uploading and ordering new copies - then found out that the cover issue was caused by a temporary bug on Lulu affecting projects with covers created in the old cover wizard (which I find is the most usable one), which were being processed with the wrong size dimensions. The fix is scheduled for early next week. I needn't have spent a day re-working the cover and uploading new versions...

Did I learn a lot? Yes.

Am I glad I had to do all this extra work? No. I wish I could just have used the working version I started with. I also have to face a much more complicated procedure whenever I make an edit to 2000 Tunes in future! I look forward to the day when PoD becomes a lot more streamlined, but until then, we live and learn. I hope some of those tips are useful to anyone else in a similar position to me.