2 MODULE DEVELOPMENT
BisQue Modules are analysis extensions to the bisque system that allow users to incorporate their own custom analysis scripts written in
Python, C++, Java, MATLAB, etc.that perform high-end computations such as deep learning based methods for use by the system and others.
One of the main reasons to convert your source code into a BisQue module is for reproducibility. Researchers in the field have exceptional work, but poor coding practices hinders the widespread adoption and justification of their findings. This is by no means their fault, but luckily help is here.
Module Files. There are six files that are needed for a module to be built and deployed on the BisQue platform.
How to Begin Building Modules
STEP 1. Source Code Prep
Building modules requires, of course, source code. Your first order of business is to upload your code to GitHub so you have a copy of your entire codebase in the event rm -rf * happens. Next, verify that your source code and README are clear enough for someone to run and understand. Do not skip this step. I have provided an example README here to help you get started. You solved a problem, now share the solution!
Once you have working source code that others have tested to make sure it runs, you can begin the process of building your very own BisQue module.
STEP 2. Fill Out the Module XML
This file should tell the user what to input. For example, if your module segments pictures of cats and dogs, you should probably have something like this:
<tag name="accepted_type" value="image" />Similarly, you should define your output to be what you are outputting. For example, if the output is a segmented image of a dog, you should have something like this:
<tag name="Segmentation" type="image">The main purpose of the XML is to put your focus on filling out relevant information instead of learning HTML. BisQue uses the information in the XML to build the user interface for your module so you don’t have to.
STEP 3. Write the Dockerfile
Probably the most important part to your module—the Dockerfile. Feel free to use example Dockerfiles from the community to get started. The main tasks you need to do are define your working directory as module and COPY all the necessary files needed to run your source code:
WORKDIR /module
COPY predict_strength.py /module/
# COPY ./source /module/source <--- If you have a source folderThere is plenty to do while creating and building the Dockerfile so please read the full description carefully. The best tip we can give you is try to replicate your current setup where your code ran successfully. If you are using Ubuntu 16.04, define that as your base image and continue to build your Dockerfile from there. Now, do not go crazy and add every pip install package you have on your system. Remember, keep your container as light as possible. Only install the necessary packages needed for your source code to run in the container.
STEP 4. Modify the Python Script Wrapper
Similar to how you would run your source code locally, the PythonScriptWrapper.py will help communicate the input from and output to BisQue. The main change to make is what file does the module need to run for the analysis. For instance, let’s say I have a predict_strength.py file with a predict function that will do everything. Then I would simply import the file and modify the outputs variable to be:
outputs = predict( bq, log, **self.options.__dict__ )Feel free to add any logging and catches you wish while diagnosing your module. This script will be your main point of contact when attempting to pinpoint missing files and dependencies, Python related issues, and many more. Look for the PythonScript.log file when testing your module.
STEP 5. Modify the Runtime Config File
Give a name to the module docker container. The other arguments can be left alone. Use this as a means to control versioning. For instance,
docker.image = cellseg-3dunet-v2 # ---> cellseg-3dunet-v{}