Skip to content

In artist core, remove _store field and flatten nested fields - #176

Open
fettuccinae wants to merge 8 commits into
metabrainz:masterfrom
fettuccinae:remove-store
Open

fettuccinae wants to merge 8 commits into
metabrainz:masterfrom
fettuccinae:remove-store

Conversation

@fettuccinae

@fettuccinae fettuccinae commented Jun 29, 2026 •

Copy link
Copy Markdown

Continuation of #172 and #175.

This PR incorporates the changes introduced in the configset PR into the indexer.

Earlier we were using convert.convert_artist to build store (an XML blob), which was then deserialized by response writers.

Changes

A new SearchField argument objconverter is added which builds a field's value from the entity object itself.

SearchArtist now no longer emits _store, every path that existed only as extrapaths is now moved to field.

tag is now represented by two parallel arrays as each tag has its tag_count.

aliases becomes alias_json, built from the alias entity's field values by tfs.aliases_to_json.

Effect

The response writer now builds the MMD Artist object from these fields, mirroring the logic in wscompat.convert_artist.

@reosarevok

Copy link
Copy Markdown
Member

Please provide commit messages :) (6249b26 for example should have a message explaining how set caused the issue and list fixes it, for future reference).

…oesnt show up in search results.

In query_result_to_dict, we are currently using a tempvals set to store the value of a field. This works well for fields where duplication is redundant but in the case of same number of mediums.track_count,
duplicate values are removed and the wrong number of tracks is sent for indexing.
We can fix this by using a list to store tempvals,
and after transformfuncs are applied, its de-duplicated(trackmediums is an exception) and added to the dict.

Changed tempvals in query_result_to_dict from set to a list.
Modified test data to test for release with same number of tracks on 2 mediums.
Modified transformfuncs to be compatible with list.
add preserve_og flag to fields, add all retreivable fields to searchFields, add index_partialdate_to_string method
@fettuccinae

fettuccinae commented Jul 6, 2026 •

Copy link
Copy Markdown
Author

Hey, sorry, I forgot to update the description of this PR.
Those commits were changes from a previous PR.

I have copied the PR description into commit message, will make sure to add them from here on.

@fettuccinae fettuccinae changed the title Preserve empty values in multi valued fields In artist core, remove _store field and flatten nested fields Jul 6, 2026
…tructor

Instead of creating parallel arrays for nested fields(here alias list), We can create a JSON string for it. objconverter on that particular nested field will take the  object and returns a list of JSON strings for the nested field.
When comparing with previous configset result, I found that the differences was in tag_count when 0, didnt show up and gender names were capitalized
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants