Bugzilla – Bug 4699
GD crashes slimserver when bad JPEG header encountered
Last modified: 2010-09-17 09:54:48 UTC
first, my apologies if i do this wrong, this is my first attempt at issuing a bug report. i am using the 21 jan 07 6.5.1 nightly release. i have done a full clear and rescan of my library. when i click to see my albums, in artist-year-album view, i don't have a problem til i get to T. TUV crashes SS, it stops and shutsdown. WYZ and the other letters work fine. i think it has something to do with the Various artists type albums, and soundtracks, etc... oddly enough, if i click "artists" first, and then "Various Artists" i can see what SS lists in artists-year-album view fine, (although its categorizations of what should be there is way off, another different bug i'm sure) i probably have not tried things you think obvious. try me. SS reports that my music library contains 1636 albums with 20210 songs by 545 artists. I think this is close but not exact, (except for the artist amount which i think is way off). i have artwork for 95% or so of my albums, and it is stored in the folder with the album, (not the tag). most art obtained with WMP, except for a very few, and some of those few don't render properly to my set size, (140 pixels) but still do render. i made my rips (all mp3) with EAC and lame 3.96 or higher. i get this in app event viewer: The description for Event ID ( 0 ) in Source ( Application ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: Perl interpreter failed. it happens EVERY time i click T. something is amiss. -mdw
please don't preset targets. they should be left undefined until they can be reviewed. As there is an official release of 6.5.1, and nothing going on with 6.5.1 nightly builds, it would be worth sticking with the official release now. I seem to recall this has been discussed at length in a forum thread, so you may want to add the link to it here, lest the same old points be done again.
i wouldn't have preset a target if i had known i was not to... i did ask what the protocol was in the aplhapage marks missed bug report, and no one replied. also, since 6.5.1 went official, perhaps someone should kill this: http://www.slimdevices.com/downloads/nightly/latest/6.5/ since its what i have just downloaded and installed. or should everyone have to check the infrequently updated official release page first every time they get and DL a nightly build? in fact, which is newer? the 21 jan 07 release, or the official release? or are they the same? and i don't recall this ever being discussed (by me anyway) in a forum thread. you're probably thinking of the alphapage bug i refer to above, and that was closed and i was told my problem shouldn't be discussed there anymore, this prompting my issuing of this bug here. https://bugs-archive.lyrion.org/show_bug.cgi?id=3255
sorry, I'll have to leave you to Chris.
thats ok, i didn't expect you'd help me anyway, once done correcting me. i am curious however, if actual SD employees are going to address the issue?
We will certainly address it. However, I should note this is the bug database, and is operated by our QA team. For quick response, you should communicate with our customer support team at support@slimdevices.com!
Well chris, i know you're still kinda new, and i appreciate the response, but i am reporting a bug, which is what this is. this is not some problem with something i'm doing, as i am doing nothing beyond normal usage of the pgm. even if (for example) the culprit is some bad artwork file paticular to me, the pgm should not respond by shutting itself down as a result. that (or whatever it may be) is a bug. and frankly, after my last attempt to get my issues (regarding internet streaming) resolved at the above support email, i really don't know why i should try that deadend again. i can fwd you the underwhelming email responses i got, if you care to see them. regardless, do u disagree that this is much more a bug issue, much less a support one? why? also, can u address the nightly build situation? should i be using the 6.5.1 official release, or the 21 jan 07 one i have? and which one is newer? (i have had this issue for months now btw) in any case, i am willing to try whatever, just tell me exactly what you'd like me to try.
Since it is being ignored in the forum, I'll reiterate my request here. Please try to narrow down the TUV section of files, if you can manage to locate a particular album that is causing the crash on page load, we can hopefully experience the same problem on other systems and tackle the problem to ensure the server no longer crashes. Well, more to the point, prevent the perl interpreter from getting whatever data is causing it to fail. Please try to understand that if none of us can replicate the problem, there is no possible way to fix this except as an accidental side effect of something else.
Created attachment 1790 [details] the artwork file i think was crashing my SS the artwork file i think was crashing my SS
kdf, i attached the file to this bug that i believe was crashing my SS. i wouldn't have found it and so on without your help and input. you have my thx and appreciation. however, it distresses me you write this in this bug thread: "Since it is being ignored in the forum, I'll reiterate my request here." not true. first of all, i ignored nothing. secondly, give me a CHANCE to try it before you claim its ignored, this post here was written less than 14 hours after u first suggested it in the forum. thirdly i do not spend all my time with SS, i do have other things to do and take care of. and don't read into that comment, i'm not implying or insinuating anything about anyone else, i am merely saying those are my circumstances. its not as if your suggestion was a two minute task in and of itself, it was not easy to do. "Please try to understand that if none of us can replicate the problem, there is no possible way to fix this except as an accidental side effect of something else." please don't insinuate that i ever had trouble understanding that or even that i didn't know it, or that i gave cause for anyone to think so. all i wanted was suggestions as to what to try, and kudos to you for figuring it out. thx again. now the ? is imo, why does that file crash SS? i do tell it to use "Folder.jpg" for thumbs, and use 140 pixel size. is it the format? the dimensions? the filesize? the black and white? and even if SS doesn't like it and can't use it, why would it crash it?
Created attachment 1791 [details] this one doesn't crash SS, but doesn't resize this one doesn't crash SS, but doesn't resize, first of a few others i will post like it.
Created attachment 1792 [details] ditto, doesn't resize ditto, doesn't resize
Created attachment 1793 [details] ditto, doesn't resize ditto, doesn't resize
Created attachment 1794 [details] ditto, doesn't resize ditto, doesn't resize
Created attachment 1795 [details] ditto, doesn't resize all the attached files were pulled off the net via right clicking in my browser and saving to disk. i think on some i changed the extension from .bmg or whatever it was to .jpg, but they still worked as jpgs inxp image viewer.
if you changed the extension, they are not supported. the resize is preformed by the GD library inside slimserver and this only works on GIF or JPG. If you change the extension, from bmp from example, they are still bmp files. They should be changed. there was no insinuation. it was only a reference to another comment you made about the problem being considered low priority becuase no one else was seeing it. no on is calling you stupid, no one is attacking you here. I'll get back to you on the crashing example.
sorry, I should be more clear about the changing of artwork. Open the bmp files in an image editor (paint shop pro, photoshop, or gimp (which is free)). Save as jpg type, which will give them a jpg extension and be valid for the server to shrink. Was the first sample file embedded as artwork or as a file in an album folder? was it just cover.jpg, or something else? I have to ask because bugzilla just spits it out as attachment.cgi. thanks
so far, slimserver under linux isn't showing the problem. I have not, however, done a full rescan to have the art in the gallery view. Do you ever see crashes when viewing just the album track list with the cover image at the top? I'll also try this on a windows system, but that will have to wait until tomorrow.
i had to combine all this b/c it was too much to try to reply to otherwise: "if you changed the extension, they are not supported. the resize is preformed by the GD library inside slimserver and this only works on GIF or JPG. If you change the extension, from bmp from example, they are still bmp files. They should be changed. sorry, I should be more clear about the changing of artwork. Open the bmp files in an image editor (paint shop pro, photoshop, or gimp (which is free)). Save as jpg type, which will give them a jpg extension and be valid for the server to shrink." ok, good to know, that should work for the secondary artwork which isn't resizing... those were all *.bmp's as i recall... i actually thought IE handled the conversion to .jpg when i saved them, b/c thats when i changed the extension, from inside IE, right click, "Save Picture As..." now i know, save as whatever they are, then use something to convert to true jpg for conversion. got it. "Was the first sample file embedded as artwork or as a file in an album folder? was it just cover.jpg, or something else? I have to ask because bugzilla just spits it out as attachment.cgi." the very first file is the only one i think crashes SS. i don't think it was a bmp. i think it was snatch off the web as a jpg. based on how bugzilla lists it up there, "pjpeg" perhaps a "progressive jpg?" i don't know. but i probably just changed the filename, (to "Folder"), but NOT the .jpg extension. i use " Folder.jpg " to create thumbs for all my albums, and its also the "bigger artwork." i place Folder.jpg in the album dir, one Folder.jpg to one directory. i do not embed artwork in the tags. is this what you wanted to know? "so far, slimserver under linux isn't showing the problem. I have not, however, done a full rescan to have the art in the gallery view." i didn't try to get to the album directly b4... but i bet it only crashes in thumb gallery view b/c its only in that view SS trys to "change it" so drastically. trying it now, (after putting it back) it does come up in the track listing np, and it does appear somewhat scaled smaller than its relatively large natural dimensions. when i click on the cover from the track listing, i get the full image alone in an IE window np, seems fine. both those things make it all the more confusing. all i can think is that 140 pixels are too small for it in thumb view, i dunno what else it could be. "Do you ever see crashes when viewing just the album track list with the cover image at the top? I'll also try this on a windows system, but that will have to wait until tomorrow." i didn't just now anyway, putting the file back manually (and without a rescan required btw). "it was only a reference to another comment you made about the problem being considered low priority becuase no one else was seeing it." if u look again, i actually didn't say that or imply it, but ok, no big deal. thx for the help, -mdw
I've done a scan, and gallery view is fine here. I'll have to try tomorrow on my windows setup to see if it is a windows issue. Some explanation of how artwork is handled: Browse Albums-> gallery view images are scaled by the server to a size set in server settings->interface. The utility that does the shrinking is called GD, which is a generic library for handling jpg and gif files. Browse Album at the track level is the raw image, contrained by html width and height tags. When the image is larger than the width of the frame, a javascript resets the height and width so the image will fit. This is why you see it slightly smaller than the full size. I'm not sure of the implications of pjpeg vs jpeg, but one thing to try is to see if you can convert to standard jpg as a workaround to avoiding the crash. If this does turn out as a crasher on windows, it is looking more like it will be a bug in GD which means there is not much we can do about fixing it, other then to be aware of the dangerous cases and warn against them in the wiki. If you remove the Folder.jpg that you added here as a suspicious file, are you able to rescan and click on the TUV without a crash?
yes, thats how i figured out it was that file... in artist, year, album view, at 140 pixels, it crashes SS. without that one file in the album dir, everything works normally. i think it must be that file, (the texas son, bootleg blues guitar, first attachment). i should also mention i am using IE7. i didn't realize GD wasn't used in the track listing, that explains that. but when SS crashes, it gives a line in the graphics.pm is that file GD? (the line is listed in the other bug thread linked above i first mentioned this issue) even if its a bug in GD, my ? is why does it crash SS? if GD can't handle a certain file, shouldn't it just skip it as an exception and move on? why would it bring down all of SS? also, why would GD work it on linux, but not on windows? very odd indeed.
If you are seeing a crash that mentions graphics.pm, please attach the text of that message. The only one I recall on this topic was a "perl interpreter failed". If some sort of data confusion is causing a perl interpreter failure, that means that the very basic routines that support slimserver are basically removed completely from under SS, which is why it will crash. As for why it might be fine in linux and not windows: I don't know. I haven't really determined that yet, as I have not tested myself on windows. The perl interpreter on windows is a completely different binary, and there are also compiled elements of GD that make it different among OS's. It might still be a case of an interaction between that artwork and something else on your system alone. At least the cause is narrowed down and now removed. With luck, we'll be able to find the reason why this artwork file is so bad and avoid the problem. as a workaround now exists, I'm going to downgrade the severity to fit the definitions. Im still planning to test the art today.
aha!! we have replication. Most certainly, this Folder.jpg does indeed cause a crash, Graphics.pm line 146: my $origImage = GD::Image->$constructor($imageData); So, looks like we do have something going on when GD is first fed the image. IE7 isn't a factor as I am using firefox. I'll have to do more experiments to see if I can get a better idea of what aspect of the image is the problem. If it is the fact that it is PJPEG, we should probably look at blocking GD use for unsupported types (so that BMP gets avoided as well). I'll see what can be done so that the artwork can by browser resized by element, rather than the current global flag. cc'ing triode in case there are other suggestions
opening the image with Gimp gives an error: "Corrupt JPEG data: 2 extraneous bytes before marker 0xe0. EXIF data will be ignored" I have tried to wrap Graphics.pm, line 146 with eval, but the crash is too low level to be safely trapped by that. Anyway, at least it's something to work with. I'm no image expert, so I'm not sure exactly how to detect this corruption before feeding it to GD. I'm also not sure how to forward this problem as a bug report to the author of GD. I've got a few ideas for handling other GD errors, so I'll work on that and hope that Chris has an idea of how to proceed on the GD crash.
sweet. i'm glad you were able to replicate it. at this point, i don't think i can help any further, i'm out of my depth. but i'll def follow the thread, and will be interested to see how it turns out. thx, -mdw
np. thanks for narrowing it down. The image should be fine again if you convert it to something like gif and back to jpg again. even just to cover.gif if you want to save steps. At least then you'll have a fully working setup again. for the record, the "corrupt" data in the header seems to be "created by Accusoft Corp". I am not if that kind of crap in the header is valid, but we'll have to see what GD has to say about it. I'll reword the summary to reflect where we are on this. I can test this against the latest version of GD for windows, but I'll need Dan to build the binary elements for me.
Created attachment 1799 [details] working ver of the bad artwork i took the file that was crashing SS, and i sent it to my gf who opened it in photoshop, and then saved it as a jpeg, (even though it was already some kind of jpeg). it now works fine in SS. i asked if she did anything else with metadata and so on, and she said nope. so i don't know what changed exactly, meaning format, metadata, both, or what, but it seems that it can be altered in photoshop to usable form.
There is some sort of JFIF comment block in the header of the "bad" file, and it's gone from the working file. The jpeg standard does seem to support a comment block, beginning with FF FE and two bytes to flag the size of the comment. The bad file is using this correctly, but will need a windows-build of the latest version of GD to be able to test further. I don't have the ability to build windows binaries, so will have to wait for Dan to be able to test this, and report a bug back to GD if the problem persists.
For Dan to possibly pass the bug on to the GD people (person?). Users note the possible workaround from Mike Walsh, which is to load and resave the file in an image editor.
is anyone able to build the GD library for windows and test? Did this get reported to the GD authors?
I just ran into this issue with artwork which do not have any problem being resized on Linux/Mac. Andy built GD for Windows a few days ago, but it still doesnt' seem to be working. Opening/saving the file in an editor did the trick for me, too.
Our GD for Windows was built with libgd 2.0.34 while Linux and Mac were built with 2.0.35. Do you think that's the issue?
I have no idea. From some comments in this thread it looks like GD on Windows always had a problem. I encountered the crasher for the first time after switching from my Mac to the Windows machine.
when I looked up reporting bugs on GD, they're boilerplate was to test with the latest version before filing. If when using 2.0.35, GD crashes on jpeg files with a rare but nonetheless valid JFIF header comment, we should file a bug with GD
Andy notes that the latest version of libgd on windows is a pain in the butt to build manually. Andy will talk to the guy that built the version we're using to see if he's built the latest version.
*** Bug 6272 has been marked as a duplicate of this bug. ***
Andy: did you get a chance to ping this guy?
He is planning to build a new version after they release libgd 2.0.36.
*** Bug 6491 has been marked as a duplicate of this bug. ***
*** Bug 6538 has been marked as a duplicate of this bug. ***
*** Bug 6925 has been marked as a duplicate of this bug. ***
*** Bug 6458 has been marked as a duplicate of this bug. ***
*** Bug 7597 has been marked as a duplicate of this bug. ***
There's a workaround in place, the 'real fix' is waiting on a new release of GD.
http://www.libgd.org/Main_Page are they really this slow?
Created attachment 4017 [details] Screenshot of incorrect artwork in SqueezeCenter
Comment on attachment 4017 [details] Screenshot of incorrect artwork in SqueezeCenter Sorry, please ignore, I uploaded to the wrong bug somehow...
*** Bug 11435 has been marked as a duplicate of this bug. ***
if GD is now dead, as it doesn't seem to be developed anymore, perhaps SBS should not use GD itself anymore?
I think something like ImageMagick might be a better choice, but changing graphics libraries is a non-trivial amount of work.
understood, and obviously this bug alone doesn't make that pressing, but i worry as to the security aspects. perhaps thats a ridiculous concern, i'm no expert, but if GD can be exploited, i'm left uneasy. if u have a "big board" for planning SBS development, perhaps this shoud be on it, as well as updating java script so IE8 can resize panes?
Is this now fixed in 7.6?
Yep.
how? was GD replaced by imagemagick? great work! :)
No, I wrote a new module that uses the GD algorithm but is way faster and also runs in fixed-point mode for use on non-floating-point hardware. It also does the resizing modes from ImageMagick but that were mostly done to see if they were faster (they aren't). http://search.cpan.org/dist/Image-Scale/lib/Image/Scale.pm