marker.js 3 is the core library providing you with the actual image annotation functionality that is the hard part of the job. You get the image annotation feature and the API to control it and then it gets out of your way when it comes to the user interface.
marker.js UI, on the other hand, is a UI wrapper around marker.js 3. It provides you with a simple and easy-to-use UI for marker.js 3. It is the "3 lines of code and annotating" solution marker.js 2 users are familiar with.
The choice between marker.js 3 and marker.js UI is fairly simple:
If your goal is to fully and seamlessly integrate image annotation functionality into your application, and you want to have full control over the UI, available features, and other user experience aspects, then marker.js 3 is the right choice for you.
If you want to quickly add image annotation functionality to your application with minimal effort and without worrying about the UI, then marker.js UI is the right choice for you.
Often you may want to start quickly and gradually integrate the user experience with that of your application. In such cases, you can go one of two ways:
Use marker.js UI to get started quickly and then gradually replace the UI components with your own custom components or fork marker.js UI and adjust it to your needs.
Start with a marker.js 3 demo as the basis for your integration (you can copy/paste components from the React demo, if you are using React) and then gradually adjust those components to your needs.
Ultimately, if your goal is eventual full integration of the user experience, and you are not on an extremely tight deadline, I recommend going with the second option. It will give you more flexibility and control over the user experience and let you build out your integration gradually, rather than starting with a full feature set and then having to limit it or bending the marker.js UI code to your will.