Search preview has different results than the search page.

If you try to search for a volume by typing it into the search bar on the top of the page, you get a "dropdown" that previews your search results. You can then hit enter to complete the search and get the full search results.

Unfortunately, the dropdown preview contains (many) volumes that do not appear in the complete search results. For example, search for "batman and robin". The 2011 Volume (33 issues) appears in the dropdown preview, but once you hit "enter" it is no where to be found. (It's easier to prove this if you search only for "volumes" in the search bar options.)

This problem of missing volumes extends to the results returned by the web API's "search" resource, too.

I can provide many more examples, if desired.

@cbanack: Yeah, I noticed this yesterday as well and will be filing a bug. Sorry for the trouble; hopefully it'll be fixed soon. Apologies for the trouble.

@rorie: Cool, thanks for that. There's more examples of the same problem here, if you need (see comment 45).

@rorie: Hi, it looks like you must have reported and/or fixed this bug, because the search results are much better now!

Unfortunately, I think in the process of fixing the search, a bug from a few weeks ago has popped up again. When using the web API 'search' resource, the search terms are once again being "OR"d together instead of "AND"ed together. So when I search for "batman arkham asylum", instead of getting back just the three issues that contain "batman" AND "arkham" AND "asylum" in their names, I get back hundreds of issues, basically everything that contains "batman" OR "arkham" OR "asylum".

This problem first popped up a few weeks ago, and I reported it here. The problem was fixed shortly thereafter. But it has returned as of this afternoon, at roughly the same time that the general search started working again. Is this new behaviour (ANDing instead of ORing) intentional?

@cbanack: hmm let me see if @jslack has any input on if anything changed in API search.

@rorie: @jslack: Hey guys, any word on this issue yet (see comment 4)? Is this change to the behaviour of the web API going to be permanent?

Hey @cbanack

We've been tweaking the way results come back to best accommodate the Wiki, the author tools, the API, the autocomplete and the results page. In doing some, sometimes we get too many results, sometimes too few. Trying to find the happy medium.

When searching for 'batman arkham asylum', it's first trying to get an exact match. After that, it's trying to get a prefix match, then anything close based on distance. Failing that, it will become 'greedy' with results, and pull in a lot of stuff. In reality, it's accepting partial prefix matching, which is really lose.

The plan is to add all of the Lucene like syntax (or like google), with AND and OR support for users, and also take advantage of it on the endpoint. In your case, "AND" would be appropriate between words, but in other cases, that's not always acceptable (for example, difference between auto complete and results page).

To answer your last question, API changes are not permanent. In fact, we are building a new API (v2), with lots of new features. Search is always changing - the main thing is bringing forum search online this week.

I'm going to add some tweaks based on your feedback, and let's work together to get these results right.

@jslack: Thanks for the detailed response! I'm happy to help in whatever way I can.

I'm trying to get my own app (which uses the ComicVine web API) working again. It relies heavily on searching, but it still needs the search terms to be ANDed together. Sounds like this isn't going to be the default behaviour for the web API anymore, but I discovered that I can still get the results I want by "ANDing" the search terms manually. For example, if I change the search string "batman arkham asylum" to "batman AND arkham AND asylum", the search once again behaves the way it used to.

Can you tell me if it is "safe" for my app to rely on that "AND" syntax working in this way? That capability seems to be undocumented right now, so I'm worried that it might not be intended as a permanent feature.

@cbanack: Yes, it is indeed undocumented, and I can't guarantee it will stay that way (although I would like it to, and it probably will). Let me push a new change tomorrow and see if that helps you out. When the new v2 API is published, we'll have documentation which will indicate available search syntax, (like AND, and OR, and other literals).

I'd like to keep working with you in your test case to get the best results, while maintaining quality results elsewhere.

No problem. My app has around 1000 active/semi-active users and they are NOT shy about letting me know when there's a problem, so I tend to be pretty well-informed about issues with the web API. Let me know when you've pushed, and I'll give things a try.

@cbanack: Hey there. I've made a few changes, would you want to try some checks?

@jslack: Sure, I just ran a few tests of the web api's "search" resource. It doesn't seem like anything has changed, though?

For example, I tried searching for "batman arkham asylum". First I used this regular, unmodified query:

http://www.comicvine.com/api/search/?api_key=MY-API-KEY&format=xml&limit=100&resources=volume&field_list=name&query=batman%20arkham%20unhinged

I get back 825 results (basically any volume that has "batman", "arkham" or "asylum" in its name.)

Then I tried it again, but inserted "AND" into the string:

http://comicvine.com/api/search/?api_key=MY-API-KEY&format=xml&limit=100&resources=volume&field_list=name&query=batman%20AND%20arkham%20AND%20unhinged

I get 3 results: the three volumes named "Batman: Arkham Asylum". I think this is the ideal behaviour, so last week I modified my app to insert "AND" into all its search queries. I haven't heard any complaints about the search feature in the latest version of my app, so I can say pretty confidently that searching with "AND" is working well.

However, I think the regular search (without "AND") is returning far too many results. Perhaps it's possible to make it automatically treat the search expression as though there was an "AND" between each word? That's how the web API's search resource behaved up until last month.