The packaged version is usually installed into /usr/lib/ or /usr/share/ or something like that, whereas the installed version is in /usr/local/lib/. Usually the PATH environment variable is set to prefer /usr/bin/ over /usr/local/bin/. The only things you should need to remove are /usr/bin/magic (the launch script) and the directory /usr/lib/magic (or where ever it was put), so that when you run "magic" from the command line, the one that is found is /usr/local/bin/magic, and it will only try to load shared object library files from /usr/local/lib/magic/.