MLflow is an open source platform to manage the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry.
The diffrence between MLFlow and other alternatives
How to use it in you code?
-
Use database based mlflow server.
- In linux shell: mlflow server —backend-store-uri=sqlite:///mlrunsdb.db —default-artifact-root=file:mlruns —host 0.0.0.0 —port 5000
- In python script:
- mlflow.set_tracking_uri(“http://nodelogin02:5000”)
- mlflow.set_experiment(“lung_fun”)
-
Assign a unique ID for each run.
- record_fpath = “results/record.log”
- id = record_1st(record_fpath)
- with mlflow.start_run(run_name=str(id), tags={“mlflow.note.content”: args.remark}):
-
Record cgpu information.
- parser.add_argument(“—hostname”, type=str, default=‘None’)
- parser.add_argument(“—jobid”, type=str, default=‘None’)
- parser.add_argument(“—outfile”, type=str, default=‘None’)
- p1 = threading.Thread(target=record_cgpu_info, args=(args.outfile, ))
- p1.start()
- ……
- p1.do_run = False # stop the thread
- p1.join()
-
For ‘evaluate.py’:
- mlflow.set_tracking_uri(“http://nodelogin02:5000”)
- experiment = mlflow.set_experiment(“lung_fun”)
- args.reload_run_id = retrive_run_id(experiment=experiment, reload_jobid=args.reload_jobid)
- mlflow.start_run(run_id=args.reload_run_id) # find the trained run
- mlflow.start_run(nested=True) # start a nested run
-
Record FLOPs
-
Nested mlflow for Cross-fold validation
-
Record git version by automatic git for each experiments
-
Record time_load_batch, time_update_batch
-
Record train/valid/test loss, metrics, for each epoch
-
Record experiment ID, cpu_count, data_shuffle_seed, epochs, eval_id, batch_size, fold, gpu_name, hostname, loss_name, net_name, mode, net_parameters, outfile, pretrained, remark, workers, input_size, etc.
FAQ: Fail to connect localhost:5000 A: https://stackoverflow.com/questions/60531166/how-to-safely-shutdown-mlflow-ui