Search result thumbnails vanishing

I've been noticing this for about a week now, search result thumbnails that were previously visible are progressively disappearing randomly and showing the "NO IMG" default image instead, the larger images are available (and not recently changed) and previously the thumbnails did exist but they are just not showing any more. I see this happening for volumes, issues, characters and creators cause those are what I search mainly so it could be happening to other page types as well, it is happening for newish pages and old ones and it is affecting the main search bar as well as association search. Maybe some database corruption is not telling the search engine the right thumb urls to load? or the hard drive storing image assets is deteriorating and the files are damaged or gone?

Here is an example to search for "Bloody Monday #4", the 2012 US edition, it used to have a thumbnail and doesn't any more. I'll post more examples as I run across them.

More Examples

• Attack on Titan: Before the Fall #1

Looks like all newly created pages are also getting the "NO IMG" thumbnail instead of the user added default images thumbnail or the blond surfer thumbnail if no image was used by the editor. This behavior started between 3:32pm PST and 3:58pm PST on 5/22, the time is narrowed down by looking at what new pages were created in that window and searching each of them to see if they have a search thumbnail or not.

There also instances where no thumbnail is available even for pages that have default images, example, four of the "Most appearances" characters just show the missing image icon and when those "images" are opened in another tab XML errors show instead of images:

<Error>

<Code>NoSuchKey</Code>

<Message>The specified key does not exist.</Message>

<Key>

uploads/square_avatar/11113/111130746/3768013-hawk.jpg

</Key>

<RequestId>D0748072BB149869</RequestId>

<HostId>

Jh9rW8TM2BewN9/4HFOiQv1aPCyb+bZWi4u/VKX5Xmtk6EEnVEecDdI391T2mVfT

</HostId>

</Error>

This specific problem has been around for a lot longer.

It looks like the disappearing search thumbnails might be happening to pages after they are edited, I edited Bloody Monday #5 last night (only added associations/stubs) and now its thumbnail is gone even though it was there when I reported Bloody Monday #4's thumbnail disappearing the day before. I don't see why adding associations would cause a thumbnail to disappear though o.o?

Edit: I can confirm that editing a page does remove its search result thumbnail, doesn't matter what the edit is the thumbnail is replaced by the NO IMG thumbnail.

This looks to be accelerating, I'm running across tons of pages missing their search thumbnail including popular pages like Superman, Wonder Woman, Hulk and the X-Men to name a few.

Any chance of this being looked into?

@pikahypersaid:

Yes, it is happening for users, tv series/episodes - looks like all page types are affected :/ Wasn't sure if it was just my cache until I saw this thread. I noticed you said it is happening to recently edited pages, so with users could it be recently changed avs?

@xnahtebx@pikahyper Yes, this is being looked into. Everything is technically working fine with the updating process which is good, but we're seeing a lack of image objects being returned on certain queries, which is weird (not just in the search).

I'm on it.

Same.

@pikahyper@wolverine08 I've updated this. There was recently an update with our image service, to make better use of CDNs stuff which broke some of our image generation calls.

I've gone back and re-indexed everything, and we should be in fairly good shape (with the only exception to that being `users`. When a user updates his profile it will update his image. I'm running through a users re-index now though anyway).

Thanks for reporting on this

@jslack: I'm still seeing issues with the NO IMG thumbnail, search "Bloody Monday #" and you'll see a handful.

@pikahyper Sometimes they take a while to show up, since we rebuilt all the images. Bloody Monday works for me, but you might have to let your cache expire before you see them.

@jslack: ah, ya looks like they are coming back.

@jslack: this problem only stopped for a couple of days and it is still going strong.

Still a problem.