Configuring a project with multiple modules in Java, scala and Python
Answered
We have a project with multiple modules in Java, scala and Python. We created a Gradle project and added these modules. It is easy to manage the dependencies for the JVM-based modules using Gradle. What are the recommendations for the Python modules in this case? To be specific:
- We want to use a Python virtual environment per module, what should we push into git and how can we reach a point where other developers importing, cloning or pulling from this project have almost no configurations to do manually? Currently they have to create their own virtual environments whenever a new Python module is pulled and set it as the `Module SDK` for this module. They can then use the git-checked `requirements.txt` to import new packages. The virtual environment is git-ignored.
- Is it a good idea to use Gradle at the project level, having these Python modules?
- How can we introduce inter-module dependencies (at least of the same stack)?
Thank you in advance
Please sign in to leave a comment.
For Java/Scala dependency management I'd use Gradle and pip for Python.
To configure a Python in a Java-based module you need to add a Python facet to it (often if there are already python files in project IDE can propose you to add it right after the project has been opened).
But if you plan to use Gradle for Java/Scala dependency management I'd suggest to move all python sources into separate python modules to avoid different conflicts, since in case of a Gradle-based project/module IDE generates module .iml files based on the build.gradle configuration.
>We want to use a Python virtual environment per module, what should we push into git and how can we reach a point where other developers importing, cloning or pulling from this project have almost no configurations to do manually?
In already mentioned module's configuration files (*.iml) IDE stores only Python interpretator name, and this information is not enough to create completely independent and self-sufficient environment so that other devs opening it from another machine would not need to make any additional configuration. So one can version control module's .iml files and give system-independent names to Python interpreter used in module (e.g. not containing paths) but other developers would still need to create their virtualenv for the module and install dependencies from requirements.txt (IDE provides corresponding inspection to help with it).
So I would say that since you plan to use (and I thing it is good idea) standard dependency management tools you version only relevant to this files like requirements.txt, build.gradle. And let IDE to use that configuration to set up the project structure.
Please see Managing Dependencies about how to manage python dependencies in IDE.
P.S. For IntelliJ IDE-specific help, please use IntelliJ-based specific forums like IDEA/PyCharm/Scala.Thanks.