Skip to content

Indexer changes for flattening simple entities - #178

Open
fettuccinae wants to merge 13 commits into
metabrainz:masterfrom
fettuccinae:simple-entities
Open

fettuccinae wants to merge 13 commits into
metabrainz:masterfrom
fettuccinae:simple-entities

Conversation

@fettuccinae

Copy link
Copy Markdown

Continuation of #176

…oesnt show up in search results.

In query_result_to_dict, we are currently using a tempvals set to store the value of a field. This works well for fields where duplication is redundant but in the case of same number of mediums.track_count,
duplicate values are removed and the wrong number of tracks is sent for indexing.
We can fix this by using a list to store tempvals,
and after transformfuncs are applied, its de-duplicated(trackmediums is an exception) and added to the dict.

Changed tempvals in query_result_to_dict from set to a list.
Modified test data to test for release with same number of tracks on 2 mediums.
Modified transformfuncs to be compatible with list.
add preserve_og flag to fields, add all retreivable fields to searchFields, add index_partialdate_to_string method
…tructor

Instead of creating parallel arrays for nested fields(here alias list), We can create a JSON string for it. objconverter on that particular nested field will take the  object and returns a list of JSON strings for the nested field.
When comparing with previous configset result, I found that the differences was in tag_count when 0, didnt show up and gender names were capitalized
i.e. Annotation, Cdstub, Editor and Tag. Their fields are flattened already and have no nested objects, so the response writers can easily recreate them.
The nested alias fields becomes alias_json that the response writer unmarshalls
The nested alias fields becomes alias_json that the response writer unmarshalls. Tags are represented as 2 parallel arrays, since each tag name has its own count.
Aliases are serialized into JSON using aliases_to_json tranformfunc. aliases_to_json mirrors what convert.convert_alias_list produces
Aliases are serialized into JSON using aliases_to_json tranformfunc. aliases_to_json mirrors what convert.convert_alias_list produces
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant