JSX and EJS are indeed both template languages, however, types where the similarities end. JSX is Facebook, inviting the JS world to make recordings with your React components simpler and more efficient ( here you can learn more about JSX ).
To answer your questions in order:
ReactDOM, React, and the JSX template language are a set of tools to make javascript presentation code more enjoyable and more efficient using a sophisticated difference mechanism to calculate when DOM rebuilds are needed. EJS is similar to the JSX part of this equation, but cannot talk to your React components.
You should use JSX to write that your React components should return to the DOM in a friendlier, more readable way than what would normally be written.
They could be used in combination, but it would be a long, meaningless battle on the hill to make them talk to each other, not to mention the fact that you will need to build a translator to get the EJS templates to be understood by React and ReactDOM. I would not try.
I assume you are embarking on a React journey ... Welcome! At first everything will be confusing, but continue to dig. Once you have mastered the basics, React development is a lot of fun and a great community surrounding libraries.