I got these error messages for more than 24 hours now. Is it just me?
The problem is: The works can be added but they do not seem to be indexed.
The last six entries od this work list cannot be found. They do not seem to exist… How can that be fixed. A resub with a small change doesn’t seem to be working.
I’ve definitely been able to replicate the issue, the website does crash right after I create a new work entity.
The issue is certainly related to search indexing, although neither the website nor the search server are printing a useful error (I get pretty generic sounding mapper_parsing_exception), so I will need to investigate a bit. In the meantime, it does look like creating the works does succeed, only the indexing is broken.
I’ll run the search reindexing right now as a stopgap for those works created during the past couple of days.
I’ve isolated the issue, it’s related to a beta feature announced here.
The production and beta websites use the same search server and that caused an issue.
I’ve reverted the search indexing, which means you shouldn’t see the website crash after work creation anymore. It also means the feature is deactivated on beta until I fix the underlying issue.
I will be taking the website down shortly to try to fix the issue; downtime shouldn’t be very long.
EDIT:
I can’t seem to replicate the issue anywhere other than in production, which has made debugging the issue quite tricky.
I haven’t been able to fix it yet, but I have put in a stopgap measure to prevent from crashing the website and give me more debugging information (which has finally pointed me to the — unexpected — core of the issue).
So please be aware that although the website doesn’t crash anymore when submitting a work, the search issue is still there and new works won’t appear in the search results until the issue is fixed.
I added many works every day and none of them have been indexed correctly, but you are right. I checked some works I added a few days ago. They are indexed correctly now. The works I added yesterday are still not indexed.
So, yes they are indexed really slow. Maybe once a day manually?
As @indy133 suggested, I’ve been running the indexing manually about once a day during quiet hours to make the situation a bit more bearable for everybody
Basically it hasn’t been an easy solution at all and I’ve had to rewrite more code than I originally wanted, and it’s quite important code that is slow and tedious to test.
I’m still working on the proper fix but it should land soon™
I’ve done a slew of improvements of the search server on top of fixing the issues from this topic, most of which you probably won’t perceive (hopefully search should be a tad faster).
One visible detail is that searching for works by an author (the feature that caused the issues in this topic) is finally working properly. I am planning on soon adding the same feature for editions and edition groups based on their author credits.
Thank you for your help and patience, and please do let me know if you see anything else wrong with search results or anything related!