View more context

 

Firesphere

I've updated Solr Search to use the BufferedAdd plugin. If anyone has a really, like, seriously large dataset to test it with, please do so?

Firesphere

More info here: https://github.com/Firesphere/silverstripe-solr-search/pull/130

Show 1 attachment(s)
GitHub  
Add documents through Buffered Add by Firesphere · Pull Request #130 · Firesphere/silverstripe-solr-search

Use Buffered Add instead of building the buffer ourself Still include the option to use a group Add a commit at the end I'm not just making this PR for HacktoberFest You rock. Thanks a lot.

Hide attachment content
Firesphere

If anyone is interested, here's the "scope" of the FTS API surface, which is way too massive. I've marked it as "won't fix" because I don't believe the API surface should be that big. But, if anyone wants to, a module extending my base compatibility module that covers (some/most) of this, is fine with me https://github.com/Firesphere/silverstripe-solr-compatibility/issues/2

Show 1 attachment(s)
GitHub  
Inventory of missing APIs, potential upgrade pain · Issue #2 · Firesphere/silverstripe-solr-compatibility

I've taken a look at over 100 distinct implementations of fulltextsearch to see how inclusive this compatibility module is and where some of the sharp edges will be in practical application of ...

Hide attachment content
Firesphere

Note that I do not want or expect a PR covering all the "open" issues, because I do not feel that they should even exist in the first place.

Firesphere

I might accept a few. Note that some methods have moved to a different place instead of actually being gone. Or some things might seem unused, while they are not actually unused due to traits 🙂

null

how big is "large"? I've got the NZ street addresses dataset I could throw at it... 2M-ish records

Firesphere

That sounds large enough, as the old method ran out of memory at roughly 100k records 😄