I have a C# application which reads and writes to an ElasticSearch database using NEST. I'm storing documents of type
I'm also setting some mapping on the fly just after the creation of the index. I'm doing it via NEST calls, see below:
elastic.Map<Dictionary<string, object>>(m => m
.AutoMap().AllField(a => a.Enabled(false))
.DynamicTemplates(dt => dt
.DynamicTemplate("pv_values_template", t => t
.Mapping(m2 => m2
.Number(n => n
The call accepts an
indexName which is of course the name of my index. I'm doing this to tell ElasticSearch to set specific mappings to the
Dictionary<string, object> documents (can't be done in advance as the type is
Now, in this project I store documents on different indexes like:
and so on...
Each year a new index is created, the mapping is set and storing stuff is just fine, everytime.
Then problem arise on reading back the documents:
when I read back the documents I'm using a special index which is:
This way I can retrieve all the stuff for all the years without knowing in advance the years, just get all the data stored in the database. See below:
var dataRecordsArray = elastic.Search<Dictionary<string, object>>(searchDescriptor => searchDescriptor
query.Term(dataRecord => dataRecord["GroupId"], groupId) &&
query.DateRange(dateRangeQueryDescriptor => dateRangeQueryDescriptor
.Field(dataRecord => dataRecord["DateTime"])
.Sort(dataRecord => dataRecord.Ascending(data => data["DateTime"])))
When I'm doing this, the ElasticSearch engine is complaining that the index I selected lacks the mapping. I know that
acme_company_* is not the same string of
acme_company_2017 but shouldn't it be the same, mapping-wise?
This is the actual error I get:
No mapping found for [DateTime] in order to sort on
DateTime is a key on the
Dictionary<string, object> document. When I read back the document, if I provide the very same index (like for example
acme_company_2015) everything works just fine
Please note that looping through the indexes from a given year to the end year (and joining the results together) is not an option as the system has to be scalable. Moreover, the indexes may span a shorter time, like a month or even less
I'll answer myself here,
it turned out that when using
acme_company_* I was matching even another set of indexes I didn't mention (i.e.
acme_company_metadata) which don't have the mapping.
Solution is to match only the
acme_company_<year> indexes, excluding
acme_company_metadata. It can be done tightening the regex to match only the year