In summer 2024, I co-authored a chapter on tabular datasets , where we tested DuckDB’s limits on nested queries on big datasets. We found that it works well when enough memory is available, but using it on a local machine with such datasets does not easily work out of the box. Some code can be found here .
Earlier I also updated the instructions for automatically publishing python packages from github, using the trusted publishing method instead of secret tokens.