• Set Up Python and the LabKey Package
• Create Target Environment
• Obtain and Customize Example Script
• Define API Wrapper
• Authentication
• Run Demo Script
• Explore the Example
• select_rows(): retrieve data
• create(): create a new dataset
• insert_rows(): populate a dataset/append new rows
• More Examples
• Create and Populate a List
• Update an Existing Dataset
• Create a Container

Set Up Python3 and the LabKey Package

Check the API documentation to find out the current supported version of python. Download and install a supported version for your operating system from the Python site. If you include Python on your path (one of the options during installation) you will be able to run it from anywhere.

Documentation for setting up Python is available on GitHub:

• Python API: labkey-api-python

To test that the correct version of Python is active on your path, run this from the location where you will run your Python scripts:

python --version

Once Python is installed correctly, install the labkey package as follows, the '--upgrade' argument ensures the latest version of the package will be loaded:

pip install --upgrade labkey

Create Target Environment

To use our example script, you need a running server and a target environment with data in certain expected locations. To set up this target environment, complete these steps:

1. Log in to your LabKey Server as an admin. The example we provided expects you are building and running your own development server locally on "localhost:8080".

2. On your server, create a project called "Tutorials" if you don't already have one. Choose "Collaboration" as the project type.

3. Download this folder archive. It contains the target dataset you will use for this demo.

• PythonDemoStudy.folder.zip

• Create a new subfolder named "Python Demo Study". Choose folder type Study and accept other defaults.
• In the new study, click Import Study.
• Choose the "Local zip archive" you just downloaded (PythonDemoStudy.folder.zip) and click Import Study.
• When the import is complete, click Python Demo Study to go to the main page of your study.

Obtain and Customize Example Script

Download the attached script and place it where you will want to run it:

• labkey_api_python_example.py

Define API Wrapper

On your server, notice the URL where your Python Demo Study folder is located. The script uses details from this URL to identify the location to work on. Creating an "API Wrapper" encapsulates this access information simplifying the rest of the script. You only need to instantiate a single APIWrapper for use throughout your script to access data in the same container (folder or project).

Parameters:

• labkey_server: (Required) This is the base URL of your server, for example "www.labkey.org" to access this support site. On a development machine, it might be "localhost:8080" and on a cloud/trial server it might be something like "your_url.trial.labkey.host". If you are not using localhost:8080, edit this value in the script provided.
• container_path: (Required) The project/folder path within that server. In our example, "Tutorials/Python Demo Study". If you used a different location on your server, edit this value.
• context_path: Dependent upon the server configuration, appearing between the base URL and the project/folder path. Typically '' (None) or 'labkey'.
• This argument to APIWrapper is optional, but required when the 'labkey' in the path is present.
• If setting to None, you can also remove it from the script, and the call to APIWrapper, entirely. If you do so, be sure to provide remaining arguments using "name=value" syntax.
• use_ssl: Check whether your server requires SSL on the Site Settings page of the Admin Console. Typically this is 'True' for remote or cloud servers and 'False' for a localhost development machine where the URL starts with http (not https). This variable defaults to 'True'.
• api_key: Using an API key, or a session-specific API key that will expire after the current login session, lets you test your script without including your actual login credentials. The example script we send assumes that you have access credentials in a netrc file. Edit this line and include the argument in the API wrapper call to use the key instead. Learn more below.

labkey_server = 'localhost:8080'
container_path = 'Tutorials/Python Demo Study'
context_path = 'labkey'
use_ssl = False
#api_key='your_API_key_here'

api = APIWrapper(labkey_server, container_path, context_path, use_ssl)

If instead you used a cloud trial server named "pythondemo.trial.labkey.host", where there is no "/labkey" context path and you use an API key, this section of your script could instead read:

labkey_server = 'pythondemo.trial.labkey.host'
container_path = 'Tutorials/Python Demo Study'
context_path = ''
use_ssl = True
api_key='your_API_key_here'

api = APIWrapper(labkey_server, container_path, context_path, use_ssl, api_key)

OR you could omit the context_path argument entirely; just be sure to override the default parameter values and specify the intended parameters like this:

labkey_server = 'pythondemo.trial.labkey.host'
container_path = 'Tutorials/Python Demo Study'
#context_path -- Not needed
use_ssl=True
api_key='your_API_key_here'


api = APIWrapper(labkey_server, container_path, use_ssl=use_ssl, api_key=api_key)

Authentication

The Python script must be authenticated to access data on the server.

You can provide credentials for your own login, or generate an API key for your Python code to use to access the server as "you". If you prefer, or if you are using a trial server, you can use a session specific temporary key that will expire after the current login session.

The credentials can either be directly inserted into the Python script, or you can use a netrc file instead. Our example script expects to use a netrc file and can work with any login credential you choose.

Use a netrc file

Using a netrc file allows you to make the credential or API key available without including it in the script. Documentation is available here: Create a netrc File.

In this file, use your actual LabKey Server login and password, or to use an API key or session key (which you must do if using a cloud trial server), enter the following:

• machine: your server machine
• login: "apikey"
• password: Use the API Key or Session Key you generated as the password.

machine localhost
login apikey
password your_API_key_here

In this case, you would not provide the api_key parameter in your script call to APIWrapper:

api = APIWrapper(labkey_server, container_path, context_path, use_ssl)

Use Credential Directly

To modify the script to contain an API key or session key directly, edit the APIWrapper definition to add the "api_key" parameter, as shown below, substituting your own key.

api = APIWrapper(labkey_server, container_path, use_ssl, api_key='your_API_key_here')

Alternately, you could define the key as a variable in the script and then use it as follows:

labkey_server = 'localhost:8080'
container_path = 'Tutorials/Python Demo Study'
context_path = 'labkey'
use_ssl = False
api_key='your_API_key_here'

api = APIWrapper(labkey_server, container_path, context_path, use_ssl, api_key=api_key)

Run Demo Script

Navigate to where you placed the example script, then run it from the command line.

python labkey_api_python_example.py

Return to the Tutorials/Python Demo Study folder on your server and click the Clinical and Assay Data tab. You will see a new dataset named "Python Demo Exam With Delta From Average BMI". Click it to see the output from the example script.

To rerun this example (such as after making a few changes of your own) be sure to delete that output dataset or change the name of what will be created.

Explore the Example

Once configured and run against our target environment, you can see how the Python API has been used to work with data. Remember that we used APIWrapper to encapsulate server and context information in the "api" object. Now we can call api.query.select_rows (and similar), providing the schema and table in that location that we want to read from and write to.

select_rows()

Use select_rows() to query a specific table and get result sets to analyze locally. The example populates an array of "people" for analysis. An excerpt:

schema = 'study'
table = 'PythonDemo_ExamData'

participant_rows = api.query.select_rows(schema, table)
people = participant_rows['rows']
...

for person in people:
...

The example then iterates over the selected result, performing calculations, sorting, and preparing an analyzed result set.

create()

The example creates a new dataset for the results after analysis is performed. Review your downloaded script to see how the new dataset is defined. The definition, named "new_exam_dataset_def" here, is then passed back to create() in the same API wrapper context, so it will appear in the same folder on the same server.

new_exam_dataset_domain = api.domain.create(new_exam_dataset_def)

The "new_exam_dataset_def" includes parameters and fields that define the new dataset you see created when you run the script. These include:

• 'kind': 'StudyDatasetVisit'
• In a date-based study, you would use 'kind': 'StudyDatasetDate'
• 'domainDesign'.'name': 'Python Demo Exam With Delta From Average BMI',
• 'domainDesign'.'fields': Including names, labels and datatypes (see example for list)
• 'options':
• 'demographics': False (boolean) Whether to mark the dataset as a demographic one in the study.
• 'datasetId' : (int) Specifies a dataset ID to use, the default is to auto generate an ID
• 'categoryId' : (int) Specifies an existing category ID
• 'keyPropertyName' : (str) The name of an additional key field to be used in conjunction with participantId and (visitId or date) to create unique records
• 'useTimeKeyField' : (boolean) Specifies to use the time portion of the date field as an additional key

insert_rows()

Our example then populates the new dataset with the two sets of filtered results from the Python script calculations using insert_rows():

participant_rows = api.labkey.query.insert_rows(schema, table, oneSixtyHeightPeople)
participant_rows = api.labkey.query.insert_rows(schema, table, oneEightyHeightPeople)

More Examples

There are more actions you can perform using the Python API. To try the following additional examples, you will need be building your own development machine where you can add new modules. These extensions will not work on a cloud trial of LabKey Server.

Create and Populate a List

Uncomment the following section to create and populate a new list.

This snipped assumes you have a module (here named 'simpletest') containing a TSV file from which to populate the list. The TSV should be in the module's resources/data/ folder. Here it is named 'Priority.tsv'.

# You can also use templates to create a definition. This would create a list called "Priority" (defined in priority.tsv in the simpletest_module).
new_list_domain_from_template_def = {
    'module': 'simpletest',
    'domainGroup': 'todolist',
    'domainKind': 'IntList',
    'domainTemplate': 'Priority'
}
print("Creating new list: Priority")
new_test_list_domain = api.domain.create(new_list_domain_from_template_def)

Update an Existing Dataset

The original example creates a new dataset with new information about the participants whose data is in the original Python Demo Exam dataset. If instead you want to update an existing dataset, such as adding a new column to the original dataset and populating it with calculated data, you need to both modify the domain (definition of the dataset) and then add the data.

To use our example, uncomment this section of the script, which will create a new BMI column in the Python Demo Exam dataset, then populate it.

# This section will add a new 'BMI' column to the original Python Demo Exam dataset and populate it:
schema = 'study'
table = 'PythonDemo_ExamData'
print("Add and populate BMI column in Python Demo Exam")

PythonDemo_Exam_domain = api.domain.get(schema, table)
PythonDemo_Exam_domain.add_field({
    'name': 'BMI',
    'rangeURI': 'double'
})

api.domain.save(schema, table, PythonDemo_Exam_domain)

participant_rows = api.query.update_rows(schema, table, people)

Remember to delete the "Python Demo Exam With Delta From Average BMI" dataset created by the script (as well as the "Priority" list, unless that section of the script is commented out) before rerunning.

Several calls are demonstrated here:

• domain.get(): Retrieves the domain for the given schema and table.
• domain.addField(): Adds the new field; specify both name and type.
• domain.save(): Push the redefined domain back to the API wrapped location for that schema and table.
• query.update_rows(): Update the rows with the revised set of data the script has stored in the 'people' Notice the original call to select_rows() populated the "people" object with the participant_rows from the original dataset.

If it is not already part of the default grid view, you can add it:

The script also defines, but does not use, a calculation of the delta between the current individual's BMI and the average of all participants. To use it, you could expose the "bmi_delta_from_avg" value as a column in a similar way.

Create a Container

Unrelated to the above examples, you can use the python API to create a new container and pass through parameters as shown below. This enables scripting the creation and population of new folders, workbooks, etc.

To create a basic default container "container1" under the /Tutorials project, update this snippet ensuring you are passing the right values to APIWrapper:

# The following constants will have to be edited according to your own server values
labkey_server = 'localhost:8080'
container_path = '/Tutorials'
context_path = 'labkey'
api = APIWrapper(labkey_server, container_path, context_path, use_ssl=False)

container1 = api.container.create("container1")

There are additional parameters available that you can pass through, letting you change the type, title, and whether the new container is a workbook. The function definition for create is :

def create(
    server_context: ServerContext,
    name: str,
    container_path: str = None,
    description: str = None,
    folder_type: str = None,
    is_workbook: bool = None,
    title: str = None,
)

Next Steps

Now that you have worked through our examples, you can adapt your own Python scripts to work with data in other locations and datasets.

Related Topics

• LabKey Client APIs