Getting Started

The following example shows how to connect to browser, navigate to a specified web page, and then extract the page title.

from trio_cdp import open_cdp, page, dom

async with open_cdp(cdp_url) as conn:
    # Find the first available target (usually a browser tab).
    targets = await target.get_targets()
    target_id = targets[0].id

    # Create a new session with the chosen target.
    async with conn.open_session(target_id) as session:

        # Navigate to a website.
        async with session.page_enable()
        async with session.wait_for(page.LoadEventFired):
            await session.execute(page.navigate(target_url))

        # Extract the page title.
        root_node = await session.execute(dom.get_document())
        title_node_id = await session.execute(dom.query_selector(root_node.node_id,
            'title'))
        html = await session.execute(dom.get_outer_html(title_node_id))
        print(html)

We’ll go through this example bit by bit. First, it starts with a context manager.

async with open_cdp(cdp_url) as conn:
    ...

This context manager opens a connection to the browser when the block is entered and closes the connection automatically when the block exits.

Although we are now connected to the browser, The browser has multiple targets that can be operated independently. For example, each browser tab is a separate target. In order to interact with one of them, we have to create a session for it.

# Find the first available target (usually a browser tab).
targets = await target.get_targets()
target_id = targets[0].id

The first line here executes the get_targets() command in the browser. Trio CDP multiplexes commands and responses on a single connection, so we can send commands from multiple Trio tasks, and the responses will be routed back to the correct task.

Note

We didn’t actually specify which connection to run the get_targets() command on. The async with open_cdp(...) context manager sets the connection as the default connection for all commands executed within this Trio task. When you run any CDP command inside this task, it will automatically be sent on that connection.

The command returns a list of TargetInfo objects. We grab the first object and extract its target_id.

# Create a new session with the chosen target.
async with conn.open_session(target_id) as session:
    ...

In order to connect to a target, we open a session based on the target ID. As with the connection, we do this inside a context manager so that the session is automatically cleaned up for us when we are done with it.

async with session.page_enable():
    async with session.wait_for(page.LoadEventFired):
        await page.navigate(target_url)

This code block is where we actually start to manipulate the browser, but there’s a lot going on here so we’ll take it one line at time, starting from the bottom and moving upward.

On the last line, we call await page.navigate(...) to tell the browser to go to a specific URL. However, we don’t want our script to continue executing until the browser actually finishes loading this new page. For example, if we try to extract the page title before the page has loaded, it will fail!

The solution is to wait for the browser to send us an event indicating that the page is loaded. In this case, we want to wait for page.LoadEventFired. The async with session.wait_for(...) block means that the block will not exit until the event has been received.

However, the browser does not send page events unless we enable them. In raw CDP there are commands page.enable and page.disable that turn page events on and off, respectively. But in Trio CDP, we make this a little cleaner by once again using a context manager. The async with session.page_enable() block will turn on page events, run the code inside, and then turn off page events automatically.

Note

If another task is using page events concurrently, the context manager is smart enough not to disable page events until all tasks are finished with it.

root_node = await dom.get_document()
title_node_id = await dom.query_selector(root_node.node_id, 'title')
html = await dom.get_outer_html(title_node_id)
print(html)

The last part of the script navigates the DOM to find the <title> element. First we get the document’s root node, then we query for a CSS selector, then we get the outer HTML of the node. This snippet shows some new APIs, but the mechanics of sending commands and getting responses are the same as the previous snippets.

A more complete version of this example can be found in examples/get_title.py in the repository. There is also a screenshot example in examples/screenshot.py. The unit tests in tests/ also provide some helpful sample code.