The autoremove is just packages that were not specified, but were installed for packages that were dependencies for other packages. And either the original package was removed or the dependency changed.
/var/log/dpkg.log* has the history of packages added/removed. But you can also simply note which packages are being removed, and if you find you are missing one, you can reinstall the package.
These are just apt packages (so not pip or github manually installed software).
Here is the exert from the man page:
autoremove (apt-get(8))
autoremove is used to remove packages that were automatically installed to satisfy
dependencies for other packages and are now no longer needed as dependencies
changed or the package(s) needing them were removed in the meantime.
You should check that the list does not include applications you have grown to
like even though they were once installed just as a dependency of another
package. You can mark such a package as manually installed by using
apt-mark(8). Packages which you have installed explicitly via install are also
never proposed for automatic removal.
Just be aware of what you install/remove. I prefer to keep it clean, but you can also mark the packages you want or just reinstall them.
To mark a package as ‘manual install’ (and no longer show up on auto remove):
sudo apt-mark manual <package_name>
Mark
