Repository navigation
Indexer changes for flattening complex entities - #179
Open
fettuccinae wants to merge 22 commits into
Open
fettuccinae wants to merge 22 commits into
fettuccinae wants to merge 22 commits into
Conversation
…oesnt show up in search results. In query_result_to_dict, we are currently using a tempvals set to store the value of a field. This works well for fields where duplication is redundant but in the case of same number of mediums.track_count, duplicate values are removed and the wrong number of tracks is sent for indexing. We can fix this by using a list to store tempvals, and after transformfuncs are applied, its de-duplicated(trackmediums is an exception) and added to the dict. Changed tempvals in query_result_to_dict from set to a list. Modified test data to test for release with same number of tracks on 2 mediums. Modified transformfuncs to be compatible with list.
add preserve_og flag to fields, add all retreivable fields to searchFields, add index_partialdate_to_string method
…tructor Instead of creating parallel arrays for nested fields(here alias list), We can create a JSON string for it. objconverter on that particular nested field will take the object and returns a list of JSON strings for the nested field.
When comparing with previous configset result, I found that the differences was in tag_count when 0, didnt show up and gender names were capitalized
…ag_name and type_name
fettuccinae
force-pushed
the
complex-entities
branch
from
August 13, 2026 19:53
5ee4c2e to
10b9ede
Compare
i.e. Annotation, Cdstub, Editor and Tag. Their fields are flattened already and have no nested objects, so the response writers can easily recreate them.
The nested alias fields becomes alias_json that the response writer unmarshalls
The nested alias fields becomes alias_json that the response writer unmarshalls. Tags are represented as 2 parallel arrays, since each tag name has its own count.
Aliases are serialized into JSON using aliases_to_json tranformfunc. aliases_to_json mirrors what convert.convert_alias_list produces
Aliases are serialized into JSON using aliases_to_json tranformfunc. aliases_to_json mirrors what convert.convert_alias_list produces
Aliases and area_links relationships are now serialized to JSON using the aliases_to_json and area_relations_to_json transformfuncs. area_relations_to_json mirrors what convert.convert_area_relation produced with direction hardcoded to backward.
artist_links and release_links are serialized into a JSON string using url_relations_to_json transformfunc. url_relations_to_json mirrors convert.convert_artist_relation_list and convert.convert_release_relation_list
aliases, artist_links and recording_links relationships are serialized to JSON string using aliases_to_json and work_relations_to_json transformfuncs. work_relations_to_json mirrors convert.convert_artist_relation_list and convert.convert_recording_work_relation_list.
aliases, area_links, artist_links, place_links relationships are serialized to JSON string using aliases_to_json and event_relations_to_json transformfuncs. event_relations_to_json mirrors convert.convert_event_area_relation_list, convert.convert_artist_relation_list and convert.convert_place_relation_list.
artist_credit and release_list are serialized to JSON string using recording_artist_credit_to_json and recording_releases_to_json transformfuncs. recording_artist_credit_to_json uses a helper function _artist_credit_dict which mirrors convert.convert_artist_credit and convert.convert_name_credit logic to produce an artist_credit_dict. recording_releases_to_json uses helper functions _artist_credit_dict which mirrors convert.convert_artist_credit, _release_group_for_release_dict which mirrors convert.convert_release_group_for_release, _medium_from_track_dict which mirrors convert.convert_medium_list_from_track and _release_event_dict which mirrors convert.convert_release_event_list. Removed tracks.medium.cdtocs.id, tracks.medium.release.country_dates.date_day, tracks.medium.release.country_dates.date_month, tracks.medium.release.country_dates.date_year because they aren't being used anywhere.
Removed artist_credit.artists.artist.aliases.* because convert_artist_credit is being called with include_aliases=False in convert.convert_release Removed tracks.medium.release.country_dates.date_* because they aren't being used in convert.convert_release_event_list
fettuccinae
force-pushed
the
complex-entities
branch
from
August 16, 2026 13:10
fca5a4d to
530c918
Compare
fettuccinae
force-pushed
the
complex-entities
branch
from
August 16, 2026 13:11
530c918 to
38c3dc2
Compare
fettuccinae
force-pushed
the
complex-entities
branch
from
August 17, 2026 11:21
3a9b694 to
48dc686
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continuation of #178.
There is a bug upstream where
*attribute_type.gidand*attribute_type.nameare getting skipped completely.To avoid false positives while comparing with the old output, I have removed these fields from SearchEntities.
We can add them back once the bug upstream is fixed.