Biologics: Electronic Lab Notebooks
Create a Notebook
ELN: Frequently Asked Questions
Biologics: Electronic Lab Notebooks
Within LabKey Biologics, use integrated and data-aware Electronic Lab Notebooks
for documenting your experiments. These electronic lab notebooks have direct access to the data you are storing and are a secure way to protect intellectual property as you develop biotherapies.
from the main menu within Biologics.
On the dashboard, you will see summary cards for your most recently viewed notebooks, a summary of actions for you, and many ways to filter and find specific notebooks of interest. Learn more in this topic: Notebook Dashboard
To enable or disable email notifications, select Settings
from the user menu. Use the checkbox to subscribe:
- Send me notifications when notebooks I am part of are submitted, approved, or have changes requested
When subscribed, you will receive email when notebooks you author are submitted, rejected, or approved. You will also receive a notification when you are assigned as a reviewer of someone else's notebook.
Emails sent will provide detail about the nature of the notification:
- When a user starts a thread on a notebook:
- The email subject field will follow the pattern: <USERNAME> commented in <NOTEBOOK NAME>: "<FIRST FIVE WORDS>..."
- Recipients include: notebook author(s)
- The thread author will not be in the notification list. If the thread author happens to be the same as the notebook author, this author will not get an email notification.
- When a user comments on a thread:
- The email subject field will follow the pattern: <USERNAME> replied in <NOTEBOOK NAME> on thread: "<FIRST FIVE WORDS OF ORIGINAL THREAD> ..."
- Recipients include: notebook author(s) and the person who started the initial thread.
- The current commentator shall not be in the notification list. If the commentator happens to be the same as the notebook author, this person will not get an email notification.
- If a user's Messages setting at the folder level is set to "Daily digest of my conversations", then:
- The individual email subject headers within the digest will follow the pattern: "<USERNAME> commented [...]" as specified above.
- The subject-line of the digest email itself will not include comment details.
- A notification will be included in a user's digest if they have participated in the thread, if they are on the notify list, or if they are among the authors of a notebook that has been commented on.
Learn more and see a preview of the ELN product by viewing the recording of the second day of the LabKey User Conference, starting around the 29 minute mark:
Using the Notebooks
dashboard, you can track multiple simultaneous notebooks in progress and interactions among teams where users both author their own work and review notebooks created by others.
On the dashboard, you will see:
- Your Recently Accessed: Summary cards for your most recently viewed notebooks. Click any card to see the full notebook.
- For You: A summary of actions for you, with alert-coloring and the number of notebooks you have:
- Overdue, waiting for you
- Returned to you for changes
- Awaiting your review
- Submitted by you for review
- Filter Notebooks: Use many ways to filter and find specific notebooks of interest.
- All Notebooks: All notebooks in the system, with any filters you apply narrowing the list.
There are many ways you can filter notebooks to find what you need:
- Title, Description, or ID: Type to search for any word or partial word or ID that appears. When you enter the value, the list of notebooks will be searched. Add additional searches as needed; remove existing ones by clicking the 'x' in the lozenge for each.
- Created: Enter the 'after' or 'before' date to find notebooks by creation date.
- Authors: Type ahead to narrow the list of authors. Click to select. Add additional authors as needed to search; remove a filter by clicking the 'x' in the lozenge.
- Reviewers: Type ahead to narrow the list of reviewers. Click to select. Add additional reviewers to the filter as needed; remove a filter by clicking the 'x' in the lozenge.
- Review Deadline: Enter the 'after' or 'before' date to find notebooks by review deadline.
- Projects: Type ahead to narrow the list of projects. Click to select. Add additional projects to the filter as needed; remove a filter by clicking the 'x' in the lozenge.
- Status: Check one or more boxes to show notebooks by status:
- Any status
- In progress
- Returned for changes
All Notebooks Listing
The main array of notebooks on the page can be shown in either the Card View
or Grid View
. You'll see all the notebooks in the system, with any filters you have applied
Switch to the Grid View
to see details sorted and filtered by your choices.
Create a Notebook
This topic is under construction.
Notebooks give you a place to collaboratively record your research. You can have as many notebooks for as many projects as your team requires.
Create New Notebook
To create a new notebook, click Create New Notebook
from the Notebooks
- Enter a Name.
- Select the Project from the dropdown. Whether this field is required or optional is controlled by an administrator setting.
- If you are an administrator, you can click Create new project to add a new project to the system.
- Enter a Description.
- Select Start and end dates.
- By default, You are an author. Use the selection menu to add more Co-authors.
- If you have templates defined, you can choose one to use by clicking Browse Templates.
Notebook ID Generation
When you create a new notebook, a unique ID value is generated for it. This can be used to help differentiate notebooks with similar names, and cannot be edited later. The format of this ID is:
For example, the user creating the notebook here has assigned User ID "1003", the date it was created was June 21, 2021, and it is the first one created in this project.
Create New Project
During notebook creation, the author can select an existing project. If that author is also an administrator, they have the option to click Create new project
to add a new one.
- Give the project a name, description, and set a color.
- Click Create Project.
Learn more about projects in this topic:
Add Custom Fields
You can define and use custom fields for your notebooks to support additional classification that is meaningful to your team. Select the Custom Fields
tab and use Add custom field
to select an existing field to add to this notebook. Type ahead to narrow the list of existing fields to find the one you want. Shown here, the "ColumnTemperature" field is added and you can now provide a value for this field for this notebook.
To manage the set of custom fields available in your application, click Manage custom fields
. In the popup you can see the existing defined fields and how many notebooks are currently using them. Here you can also add new fields, and edit or remove any fields not currently in use. Click Done Editing Fields
Rename a Notebook
A notebook author can select Rename
menu next to the name. Non-authors will not see the Rename
option on this menu. While the system does not require names to be unique, you will want to choose something that will help your colleagues identify it on lists and dashboards.
Copy a Notebook
Once you have created and populated a notebook, you can copy it to create a new notebook with the same details. This is similar to creating a new notebook from a template
, except that templates do NOT include the value of any custom fields, and copied notebooks do include these values. You do not need to be a notebook author to make a copy of it.
Give your new notebook a name, select a project (the project of the one you copied is the default), and click Yes, Copy Notebook
to create the new one.
Add to a Notebook
A notebook lets you record your work in a series of entries
, each of which can have a custom name, span multiple days, include references to data in the Biologics application, and support entry-specific comment threads.
As you complete your notebook, everything is saved automatically. Note that refresh is not continuous, so if you are simultaneously editing with other authors, you may need to refresh your browser to see their work.
A new empty notebook looks like this:
The header section lists the name and ID, the authors, shows the project (if any), and shows the status. A details panel to the right summarizes references for all entries, may include attachments, and can be hidden by clicking Hide details
(and shown again by clicking show details).
Add to an Entry
The New Entry
panel is where you can begin to write your findings and other information to be recorded.
The formatting header bar for an entry includes:
- Insert: Select to add:
- Styling menu: Defaults to Normal and offers 3 heading levels
- Special characters: Add mu, delta, angstrom, degree, or lambda characters
- Bold, Italics, Underline
- : Link selected text to the target of your choice
- / : Make a text selection a super- or sub-script
- Text color. Click to select
- Alignment selections: Left, center, or right alignment; indent or dedent
- Numbered or bullet lists
- : Clear formatting
Rename an Entry
next to the "New Entry" title to rename it.
Add a Reference
Within the entry panel, you can use the Insert > Reference
menu, or within the text, just type a '/' forward slash to reference
anything in the Biologics registry.
The / menu lets you select the category you want to reference:
If you click Samples
you will next be able to click one of the existing Sample Types
, then can type ahead to Search
within that category, or the most recent entries in that category are shown.
Once added, the reference appears as a color-coded lozenge in the text, and is also added to the Referenced Items
list in the details panel.
Add References in Bulk
After selecting the type of reference you want to add, you can click Reference multiple [type of reference]
In the popup, you can:
- Paste (or type) your references into the box, one per line. Use sampleIDs as shown below, for example.
- Click Upload file to upload a file of references, supported formats include .csv, .tsv, .txt
- In the case of samples, assays, media, or registry entity references, click Change [category] to switch types within that category of reference.
Click Add References
to add them.
If any pasted or uploaded references cannot be found, you'll see a warning listing the invalid references (valid references are not shown in the warning.
You can either:
- Correct any errors and click Recheck References
- Or click to Skip and Add References.
Add a New Day
Place the cursor where you want to add a marker for a new date. Select Insert > New Day
. A date marker
will be added to the panel for today's date.
Hover to reveal a
delete icon. Click the day to open a calendar tool, letting you choose a different day to reference. Record activities for that day below the marker.
Comment on an Entry
Click Start a thread
to add a comment to any entry. Each entry supports independent comment threads, and an entry may have multiple independent comment threads for different discussions.
By default you will enter comments in Markdown Mode
; you can switch to Preview
mode before saving. Other dashboard options include bold, italic, links, bullets, and numbered lists. Click the
button to attach a file to your comment.
Type and format your comment, then click Add Comment
Once your comment has been saved, you or other collaborators can click Reply
to add to the thread. Or Start a thread
to start a new discussion.
For each thread, there is a
menu offering the options:
- Edit comment (including adding or removing files attached to comments)
- Delete thread
Attachments, such as image files, protocol documents, or other material can be attached to the notebook.
To add an attachment to an entry, select Insert > Attachment
. To add to the notebook as a whole, click the Attachments
area in the Details panel on the right (or drag and drop attachments from your desktop).
Add a Table
You can add a table directly using the Insert > Table
menu item, directly entering content. You can also paste directly from either Google Sheets or Excel either into that table, or into a plain text area and a table will be added for you.
Adjust Wide Tables
After adding a table to an ELN
, you can add/remove columns using the
table tools menu. Drag column borders to adjust widths for display.
When configured, you'll be able to export to PDF and adjust page layout settings
to properly show wider tables.
Entry Locking and Protection
Many users can simultaneously collaborate on creating notebooks. Individual notebook entries are locked while any user is editing, so that another user will not be able to overwrite their work and will also not lose work of their own.
While you are editing an entry, you will periodically see that it is being saved in the header:
Other users looking at the same entry at the same time will be prevented from editing, and see your updates 'live' in the browser. A lock icon and the username of who is editing are shown in the grayed header while the entry is locked.
If another user attempts to edit an entry that is either actively being edited by another author OR has been updated since they opened it (or refreshed their browser), they will be prevented from making further changes. They will see a message reading "Unable to update this notebook entry. The entry you are attempting to edit appears to be out of date. Reload the page and try again."
Your notebook can contain as many entries as needed to document your work. To add additional panels, click Add Entry
at the bottom of the current notebook. You can also add a new entry by copying an existing entry.
You cannot completely delete an entry in a notebook. Archiving an entry collapses and hides the entry.
- You can immediately Undo this action if desired.
Once multiple entries have been archived, you can display them all again by selecting Archive > View Archived Entries
at the top of the notebook. Each archived entry will have an option to Restore entry
Return to the active entries using Archive > View Active Entries
Create a duplicate of the current entry, including all contents. It will be placed immediately following the entry you copied, and have the same name with "(Copy)" appended. You can change both the name and position.
Select to open a panel where you can drag and drop to rearrange the entries. Click Save
in the popup to apply the changes.
Export Notebook as PDF
To export a notebook as a PDF, you must have correctly configured the Puppeteer
service. Learn more in this topic: Configure Puppeteer
Submit for Review
When your notebook is ready for review, click Submit for Review
You'll review your entries, references, and attachments to confirm you have included everything necessary, then click Go to Signature Page
Select one or more reviewers, provide a deadline, and comments if needed. You'll sign with your display name or email (click Configure name
to set the display name to use). Check the box to certify that the data contained in the notebook is accurate. Click Submit Signed Notebook
Learn more about the notebook review process in this topic:
Notebooks can be organized and color coded by project, helping users group and prioritize their authoring and review work. New projects can be added during notebook creation.
From the main menu, click Notebooks
, then select Manage > Projects
to open the dashboard.
Projects listed here will be available for users adding new notebooks.
Delete Unused Project
To delete projects, an administrator can select one or more rows and click Delete
. Projects with notebooks associated with them cannot be deleted.
Add New Project
To add a new project, an administrator clicks Create New Project
, providing a name, identifying color, and optional description, then clicks Create Project
Once added, a project definition and color assignment cannot be edited.
By default, new notebooks do not require a project association. An administrator can require the selection of a project for every notebook using the Settings
option from the user menu. Look for the Notebook Settings
If this setting is enabled while there are existing notebooks without project affiliations, these notebooks will display a banner message reminding the editor(s) to Add to a project
before submitting. Click the
to enable the project selection dropdown.
Select Project for Notebook
Notebook authors will see the colors and names of projects available when they create or edit notebooks:
This topic covers the process for reviewing and providing feedback to the authors on a notebook. Iteration of review and comment cycles will continue until all reviewers are satisfied with the contents of the notebook.
When you are assigned as a reviewer for a notebook, you will receive a notification and see the notebook marked as Needs Review
on the Notebook List
Click the notebook card (or name on the grid) to open it. You'll see the notebook contents, including details, entries, references, etc. When ready to begin, click Start Review
In review mode, you can check the contents of the notebook, adding Comments
to specific entries if you like, viewing attachments, checking the linked references, etc. There are three actions available:
- Close and go home: Conclude this session of review without making any decision. This notebook will remain the same status awaiting your review.
- Suggest changes and return: Click to "reject" this notebook, returning it to the submitters with suggestions of changes.
- Go to signature: Click to "approve" this notebook and proceed to the signature page.
Anyone assigned the Editor role or above can be a reviewer of a notebook.
As the reviewer of a notebook, you also have the ability to add other reviewers, or remove yourself from the reviewer list if multiple reviewers are assigned. Open the notebook and click Manage reviewers
Add reviewers by selecting them from the dropdown. Remove a reviewer by clicking the 'X' (at least one reviewer is required). Click Update Reviewers
Suggest Changes and Return (Reject this Version)
While in review mode, if you are not ready to approve the notebook, click Suggest changes and return
. Provide comments and requests for changes in the popup, then click Yes, return notebook
The authors will be notified of your request for changes. Your review comments will be shown in a new Review details
panel at the top of the notebook.
This phase of your review is now complete. The notebook will be in a "returned for changes" state, and co-authors will be expected to address your comments before resubmitting for review.
Authors Respond to Feedback
When a notebook is returned for changes, the author(s) will see the Review details
and be able to Unlock
and update the notebook to address them.
After updating the notebook, the author(s) can again click Submit for Review
and follow the same procedure as when they originally submitted the notebook
As a reviewer in "review mode", when you are ready to approve the notebook, click Go to signature
The signature page offers a space for any comments, and a Signature
panel. Uses your display name or email address and checks the box to certify that the data contained in this notebook is accurate. Click Sign and approve
The authors will be notified of the approval, and be able to view or export the signed notebook.
The notebook will now be in the Approved
Approved notebooks cannot be edited, but if desired, they may be used to generate templates
for new notebooks.
A notebook template provides a convenient way for researchers to begin their notebook with the sections and boilerplate content needed in your organization. Different types of notebooks for different purposes can be started from customized templates, making it more likely that final work meets the reviewers expectations.
Create New Template
To create a new template from scratch, click Templates
on the main menu under Notebooks
. You can also select Manage > Templates
from the Notebooks Dashboard
Click Create New Template
Check the Sharing
box if you want to create a Team Template
, shared with other users in your lab.
Click Edit Template
to begin adding the starter content you want the template to contain.
Give your template a unique name so that you can locate it later. The default name, "New Template" includes a > Rename
Continue in template edit mode to add the content you want to be included in your template. Learn about creating, naming, populating, and arranging Entry panels in this topic: Create a Notebook
Any references, attachments, and fields you include in the template will be available in any notebooks created from it. Notebook creators will be able to edit the content, so for example, you might include in a template directions for completing a "Conclusions" section, which the individual notebook creator would replace with their actual conclusions.
As you edit the template, saving is continuous. When you are finished making changes, click Lock Template
to save this version.
To share your template with your team, check the Share this template with team members
box. Team templates will be shown on a separate tab to all team members, and indicated with the
Create Template from Notebook
You can also create a new template from an existing notebook. The template will include entry content, attachments, fields, and references for repeat use in other notebooks.
Open the notebook, the
menu next to the title, then click Save As Template
Give the template a name, check the sharing box if you want to share this template with team members, then click Yes, Save Template
Create Template from Another Template
While editing a template, you can use the contents to create a new template by selecting Save As Template
from the template name
When you save the new template, you decide whether it is private or shared. The sharing setting for the new one does not need to match the source template.
Create Notebook from Template
To create a new notebook from a template, you have two options.
Starting from the template, click New Notebook From Template
You can also start from the Notebook dashboard by clicking Create New Notebook
. In the first panel, under Notebook Template
, click Browse Templates
In the popup, locate the template you want to use. Team templates are indicated with the
Proceed to customize and populate your notebook. Note that once a notebook has been created, you cannot retroactively "apply" a template to it. Further, there is no ongoing connection between the template you use and the notebook. If the template is edited later, the notebook will not pick up those changes. Likewise, if the template is deleted, the notebook is unaffected.
Create a Notebook from Another Notebook
You can create a new notebook from an existing notebook by selecting Copy Notebook
menu next to the name. Copying a notebook will include the value of any custom fields, where using a template does not preserve those values.
Give your new notebook a name, select a project (the project of the one you copied is the default), and click Yes, Copy Notebook
to create the new one.
Open the Template Dashboard by selecting Templates
from the main menu. You can also select Manage > Templates
from the Notebook Dashboard.
- My Templates lists the templates you have created, whether they are shared with the team or not.
- Team Templates lists templates created by other team members and shared with you.
Click the name of any template to open it. You can use the omnibox
to search, filter, and sort longer lists of templates.
To delete a template, select it on the template dashboard and click Delete
. You may delete templates from which notebooks were created; those derived notebooks will be unaffected.
Puppeteer is an external web service that can be used to generate PDFs from Notebooks created with LabKey ELN. To use this service, you need to obtain the puppeteer
premium module, and configure your LabKey Server as described in this topic.
Setup Puppeteer Service
puppeteer-service is a web service that allows for generation of assets (e.g. PDFs, screenshots) via Puppeteer. This service is built into a Docker image and served via Express.
This service is deployable as a Docker container on either a Linux or OSX host. Docker is required to be installed on the host machine. You can retrieve the latest Docker image from Docker Hub
using the following command:
docker pull labkeyteamcity/lk-puppeteer-service
If you would prefer to build the Docker image you can clone the LabKey puppeteer-service Git repository
and follow the README's instructions for building an image:
git clone email@example.com:LabKey/puppeteer-service.git
This image can then be run in a container:
docker run -i --init --rm --cap-add=SYS_ADMIN -p 3031:3031 --name puppeteer-service labkeyteamcity/lk-puppeteer-service
A simple HTML view has been added to allow for interacting with the service. To see this view navigate to http://localhost:3031
to get familiar.
Deploying this service on a Windows host is not supported nor tested.
POST to /pdf:
- inputURL: The URL of the website to create a PDF or screenshot from.
- apiKey: (Optional) If visiting a LabKey site that requires credentials you can specify an apiKey.
- fileName: (Optional) Name of the file that is streamed back in the response. Defaults to generated.pdf.
- printFormat: (Optional) Currently supports Letter and A4 page sizes. Defaults to Letter.
- printOrientation: (Optional) Supports Portrait and Landscape orientations. Defaults to Portrait.
Configure Puppeteer Module
- Select (Admin) > Site > Admin Console.
- Under Configuration, click Puppeteer Service.
- Set the properties following the guidance below.
The puppeteer module has two configuration modes for the puppeteer-service. These modes are:
- remote: (Recommended) Remote mode configures LabKey Server to communicate with a remote instance of the puppeteer-service. The service is treated strictly as an external web service where the LabKey Server instance does not manage the service in any way.
- docker: (Experimental) Docker mode leverages LabKey's docker module to configure and run a puppeteer-service container on the LabKey Server instance. This mode manages the lifecycle of the container (i.e. startup, running, shutdown) which shares computational resources with the LabKey server instance.
Remote mode is recommended for several reasons:
- Computational resources. Having the service running alongside LabKey Server, like it does in docker mode, can consume significant CPU and memory resources due to the overhead of the service as well as Docker.
- The remote puppeteer-service can be stood up once and shared by several LabKey Server instances (e.g. staging, production, etc).
Remote URL (Remote mode only)
The "Remote URL" property specifies the URL where the web service can be found. This is provided by whomever sets up the puppeteer-service instance.
Docker Image (Docker mode only)
Docker mode only. The Docker Image
property specifies the name of the puppeteer-service image that is available on the LabKey Server instance. This image has to be downloaded via docker. See service deployment documentation for how to pull down an image.
Docker Port (Docker mode only)
The "Docker Port" property specifies what host port the service container should be bound to.
All of the properties are also available as LabKey Server Bootstrap Properties. An example properties file (using Docker mode) would look something like:
Export Notebook as PDF
Once configured, you will see a
button in your notebooks. Click and choose Export PDF
Adjust settings including:
- Format: Letter or A4.
- Fit to Page: Check box to automatically scale down any contents wider than the page to fit the width of the PDF.
- Orientation: Portrait or Landscape.
to export to PDF.
ELN: Frequently Asked Questions
Within LabKey Biologics, you can use data-integrated Notebooks
for documenting your experiments. These electronic lab notebooks have direct access to the data you are storing and are a secure way to protect intellectual property as you work.
How can I use Notebooks for my work?
Notebooks are available in LabKey Biologics.
Who can see my Notebooks?
Every user with "Read" access to your Biologics project can also see all Notebooks created there.
What can I reference from a Notebook?
You can reference anything in your Biologics application, including data, experiments, specific samples, and other notebooks. Use a direct reference selector to place a color coded reference directly in your Notebook text. Add a single reference at a time, or add multiple references in bulk
The combined list of all elements referenced from a notebook is maintained in an Overview panel.
Learn more in this topic: Add a Reference
How are Notebooks locked and protected once signed?
After the notebook has been approved we create a signed snapshot. The snapshot will include all notebook data including:
- Notebook metadata and text
- Any data referenced in the notebook (samples, entities, experiments, assay runs, etc.)
- Attached files
The data archive will be compressed and stored in the database to allow future downloads. We will compute a SHA-2 cryptographic hash over the data archive and store it in the database. This allows us to verify that the contents of the data archive are exactly the same as the notebook that was signed and approved.
What happens to my Notebooks when I upgrade?
- Once you create a Notebook, it will be preserved (and unaltered) by upgrades of LabKey Biologics.
What are team templates?
Templates can give you a head start in creating a Notebook with a given format or containing boilerplate details. Team templates are shared with the team. You can also create personal templates that are not shared.
When you view the Templates Dashboard
, all the templates you created are listed on the My Templates
tab. Templates created by other users and shared with you are listed on the Team Templates