2019-07-23 –, Elm A
Introducting LightQuery.jl, a new querying package which combines performance with flexibility.
LightQuery.jl is a new package for querying tabular data. I'll discuss a number of things which make it special.
1) Careful use of constant propagation, so that named-tuple level operations are type stable when wrapped in a functions.
2) The ability to simultaneously collect into (and iteratively widen) several sinks at once.
3) Very few macros: only two, one for chaining and one for anonymizing. Compare the number of macros required for Query, DataFramesMeta, and JuliaDBMeta.
4) Careful tracking and propagation of line number information.
5) Huge speed-ups when sources are pre-sorted.
6) Flexibility. Row-wise operations work with arbitrary containers, provided they can be indexed out of order. Column-wise operations work for anything which has propertynames.
PhD student in heterodox economics at UMass Amherst, hobbyist programmer.