Carduus Databricks Getting Started Guide¶
Carduus is an implementaiton of the Open Privacy Preserving Record Linkage (OPPRL) specification. It allows organizations to replace personally identifiable information (PII) with enrypted tokens that can be used to correlate data records pertaining to the same subject while keeping the subject anonymous.
This notebook demonstrates the use of the carduus python package in databricks to parallelize the tokenization processes using Spark clusters.
The following is a 5 row sample dataset of PII that will be tokenized. Notice that despite cosmetic differences the first 2 rows (label = 1) are describing the same subject and will receive the same sets of token values.
In [0]:
Copied!
pii = spark.createDataFrame(
[
(1, "Jonas", "Salk", "male", "1914-10-28"),
(1, "jonas", "salk", "M", "1914-10-28"),
(2, "Elizabeth", "Blackwell", "F", "1821-02-03"),
(3, "Edward", "Jenner", "m", "1749-05-17"),
(4, "Alexander", "Fleming", "M", "1881-08-06"),
],
("label", "first_name", "last_name", "gender", "birth_date"),
)
display(pii)
pii = spark.createDataFrame(
[
(1, "Jonas", "Salk", "male", "1914-10-28"),
(1, "jonas", "salk", "M", "1914-10-28"),
(2, "Elizabeth", "Blackwell", "F", "1821-02-03"),
(3, "Edward", "Jenner", "m", "1749-05-17"),
(4, "Alexander", "Fleming", "M", "1881-08-06"),
],
("label", "first_name", "last_name", "gender", "birth_date"),
)
display(pii)
| label | first_name | last_name | gender | birth_date |
|---|---|---|---|---|
| 1 | Jonas | Salk | male | 1914-10-28 |
| 1 | jonas | salk | M | 1914-10-28 |
| 2 | Elizabeth | Blackwell | F | 1821-02-03 |
| 3 | Edward | Jenner | m | 1749-05-17 |
| 4 | Alexander | Fleming | M | 1881-08-06 |
One-time Workspace Setup¶
See the Databricks documenation for managing secrets to get started. You will need an to install the Databricks CLI and authenticate as a user that can manage Databricks secrets.
Create a secret scope for carduus encryption keys by running the following command with the Databricks CLI.
databricks secrets create-scope carduus
Add your private key as a secret in the carduus scope.
(cat << EOF
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
EOF
) | databricks secrets put-secret carduus PrivateKey
Add the secrets for the public keys of the third party organiations you will be sending and receiving tokenized data to/from.
(cat << EOF
-----BEGIN PUBLIC KEY-----
...
-----END PUBLIC KEY-----
EOF
) | databricks secrets put-secret carduus MyPartnerPublicKey
Cluster Setup¶
Add spark configuration properties to your custer by reading from the secrets created above. Your organization's private key must be stored under the carduus.token.privateKey key and all third party public keys must be stored under a key with the format carduus.token.publicKey.<profile> where <profile> is the name you would like to associate with a specific partner.
carduus.token.privateKey {{secrets/carduus/PrivateKey}}
carduus.token.publicKey.MyPartner {{secrets/carduus/MyPartnerPublicKey}}
Install carduus¶
You can install carduus on your cluster, or within the notebook. See the Databricks docs for more information.
In [0]:
Copied!
%pip install carduus
%pip install carduus
Collecting carduus
Obtaining dependency information for carduus from https://files.pythonhosted.org/packages/ff/52/f4b474670fe74e6127f63e1afbe3920b32feefb2d7f029d38e87e0712062/carduus-0.1.2-py3-none-any.whl.metadata
Downloading carduus-0.1.2-py3-none-any.whl.metadata (4.3 kB)
Collecting cryptography<43.0.0,>=42.0.5 (from carduus)
Obtaining dependency information for cryptography<43.0.0,>=42.0.5 from https://files.pythonhosted.org/packages/5f/f9/c3d4f19b82bdb25a3d857fe96e7e571c981810e47e3f299cc13ac429066a/cryptography-42.0.8-cp39-abi3-manylinux_2_28_aarch64.whl.metadata
Downloading cryptography-42.0.8-cp39-abi3-manylinux_2_28_aarch64.whl.metadata (5.3 kB)
Collecting jellyfish<2.0.0,>=1.0.3 (from carduus)
Obtaining dependency information for jellyfish<2.0.0,>=1.0.3 from https://files.pythonhosted.org/packages/a2/38/89f737cfa3c9716c583a5aac018b136d69322706cb34d3fca2c741e90533/jellyfish-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata
Downloading jellyfish-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (2.6 kB)
Collecting pyspark<4.0,>=3.5 (from carduus)
Downloading pyspark-3.5.1.tar.gz (317.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/317.0 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.3/317.0 MB 9.9 MB/s eta 0:00:32
╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.0/317.0 MB 58.6 MB/s eta 0:00:06
━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.7/317.0 MB 83.4 MB/s eta 0:00:04
━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.6/317.0 MB 138.8 MB/s eta 0:00:03
━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.5/317.0 MB 125.2 MB/s eta 0:00:03
━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.1/317.0 MB 133.7 MB/s eta 0:00:03
━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 28.8/317.0 MB 168.3 MB/s eta 0:00:02
━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 35.5/317.0 MB 184.5 MB/s eta 0:00:02
━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.3/317.0 MB 213.9 MB/s eta 0:00:02
━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.8/317.0 MB 247.6 MB/s eta 0:00:02
━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.1/317.0 MB 249.9 MB/s eta 0:00:02
━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.3/317.0 MB 165.1 MB/s eta 0:00:02
━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.5/317.0 MB 172.1 MB/s eta 0:00:02
━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.0/317.0 MB 191.1 MB/s eta 0:00:02
━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.7/317.0 MB 195.5 MB/s eta 0:00:02
━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━ 91.3/317.0 MB 192.6 MB/s eta 0:00:02
━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.9/317.0 MB 215.0 MB/s eta 0:00:02
━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━ 107.3/317.0 MB 242.3 MB/s eta 0:00:01
━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━ 114.7/317.0 MB 218.2 MB/s eta 0:00:01
━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━ 122.8/317.0 MB 234.4 MB/s eta 0:00:01
━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━ 129.9/317.0 MB 214.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━ 136.1/317.0 MB 183.7 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━ 142.0/317.0 MB 170.7 MB/s eta 0:00:02
━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 149.9/317.0 MB 216.0 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━ 158.1/317.0 MB 237.2 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━ 165.8/317.0 MB 221.6 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━ 173.8/317.0 MB 237.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━ 181.4/317.0 MB 227.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━ 190.1/317.0 MB 254.2 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━ 197.8/317.0 MB 227.5 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━ 202.9/317.0 MB 174.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━ 206.9/317.0 MB 134.2 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━ 212.5/317.0 MB 137.4 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━ 218.6/317.0 MB 168.6 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━ 224.7/317.0 MB 173.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━ 230.5/317.0 MB 173.0 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━ 237.1/317.0 MB 171.2 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━ 243.6/317.0 MB 185.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 251.7/317.0 MB 216.3 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━ 259.8/317.0 MB 231.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━ 267.2/317.0 MB 222.0 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━ 274.9/317.0 MB 218.6 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━ 286.3/317.0 MB 237.0 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━ 294.1/317.0 MB 223.6 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━ 298.7/317.0 MB 161.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━ 303.1/317.0 MB 129.0 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━ 306.4/317.0 MB 110.1 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺ 309.8/317.0 MB 115.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 313.8/317.0 MB 100.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 316.9/317.0 MB 127.8 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.0/317.0 MB 2.2 MB/s eta 0:00:00
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: cffi>=1.12 in /databricks/python3/lib/python3.11/site-packages (from cryptography<43.0.0,>=42.0.5->carduus) (1.15.1)
Collecting py4j==0.10.9.7 (from pyspark<4.0,>=3.5->carduus)
Obtaining dependency information for py4j==0.10.9.7 from https://files.pythonhosted.org/packages/10/30/a58b32568f1623aaad7db22aa9eafc4c6c194b429ff35bdc55ca2726da47/py4j-0.10.9.7-py2.py3-none-any.whl.metadata
Downloading py4j-0.10.9.7-py2.py3-none-any.whl.metadata (1.5 kB)
Requirement already satisfied: pycparser in /databricks/python3/lib/python3.11/site-packages (from cffi>=1.12->cryptography<43.0.0,>=42.0.5->carduus) (2.21)
Downloading carduus-0.1.2-py3-none-any.whl (13 kB)
Downloading cryptography-42.0.8-cp39-abi3-manylinux_2_28_aarch64.whl (3.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/3.6 MB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸ 3.6/3.6 MB 180.9 MB/s eta 0:00:01
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 96.0 MB/s eta 0:00:00
Downloading jellyfish-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (346 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/346.5 kB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 346.5/346.5 kB 59.0 MB/s eta 0:00:00
Downloading py4j-0.10.9.7-py2.py3-none-any.whl (200 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/200.5 kB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.5/200.5 kB 40.8 MB/s eta 0:00:00
Building wheels for collected packages: pyspark
Building wheel for pyspark (setup.py): started
Building wheel for pyspark (setup.py): finished with status 'done'
Created wheel for pyspark: filename=pyspark-3.5.1-py2.py3-none-any.whl size=317488496 sha256=e24365e1014504c9eef9b14500288bcb2196444e6efe80e08d090d1caacfc43a
Stored in directory: /home/spark-9f2e49af-d493-4828-9ee1-d6/.cache/pip/wheels/95/13/41/f7f135ee114175605fb4f0a89e7389f3742aa6c1e1a5bcb657
Successfully built pyspark
Installing collected packages: py4j, pyspark, jellyfish, cryptography, carduus
Attempting uninstall: cryptography
Found existing installation: cryptography 41.0.3
Not uninstalling cryptography at /databricks/python3/lib/python3.11/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-9f2e49af-d493-4828-9ee1-d6694a0c6be7
Can't uninstall 'cryptography'. No files were found to uninstall.
Successfully installed carduus-0.1.2 cryptography-42.0.8 jellyfish-1.1.0 py4j-0.10.9.7 pyspark-3.5.1
Note: you may need to restart the kernel using %restart_python or dbutils.library.restartPython() to use updated packages.
Tokenization of PII¶
In [0]:
Copied!
from carduus.token import tokenize, OpprlPii, OpprlToken
tokens = tokenize(
pii,
pii_transforms=dict(
first_name=OpprlPii.first_name,
last_name=OpprlPii.last_name,
gender=OpprlPii.gender,
birth_date=OpprlPii.birth_date,
),
tokens=[OpprlToken.token1, OpprlToken.token2, OpprlToken.token3],
private_key=dbutils.secrets.getBytes("carduus", "PrivateKey"),
)
display(tokens)
from carduus.token import tokenize, OpprlPii, OpprlToken
tokens = tokenize(
pii,
pii_transforms=dict(
first_name=OpprlPii.first_name,
last_name=OpprlPii.last_name,
gender=OpprlPii.gender,
birth_date=OpprlPii.birth_date,
),
tokens=[OpprlToken.token1, OpprlToken.token2, OpprlToken.token3],
private_key=dbutils.secrets.getBytes("carduus", "PrivateKey"),
)
display(tokens)
| label | first_name | last_name | gender | birth_date | opprl_token_1 | opprl_token_2 | opprl_token_3 |
|---|---|---|---|---|---|---|---|
| 1 | JONAS | SALK | M | 1914-10-28 | t+Yg6k4aOm5xMOMjT1nUCVbw1xM6mITKRx/APB+oU0dNo/AN2q/p20Pu2fiKd4wX5iFVK119DJAHYkFJYuI1BxgLBrzkiQKdEJKn1kMzA6k= | AWkmGR88bq2Fm/OJpcliqpYxYl43BQETABQT53Y4F8m1nJBnsPJIrQECXjC2qwiX47TI78GHDTuNNu2Sw1r6UAYILDQyFu9swKwcwf99C68= | Njlxk7k6GnIHAeV5rK43FOas0JrIB7SYd8xkRRhJikvR+OxYSmpwJyHM5tEc25+srlFOekqx6MTPDMQwUMQ/wxx33ETla7Q3po5Ypjf8Z7g= |
| 1 | JONAS | SALK | M | 1914-10-28 | t+Yg6k4aOm5xMOMjT1nUCVbw1xM6mITKRx/APB+oU0dNo/AN2q/p20Pu2fiKd4wX5iFVK119DJAHYkFJYuI1BxgLBrzkiQKdEJKn1kMzA6k= | AWkmGR88bq2Fm/OJpcliqpYxYl43BQETABQT53Y4F8m1nJBnsPJIrQECXjC2qwiX47TI78GHDTuNNu2Sw1r6UAYILDQyFu9swKwcwf99C68= | Njlxk7k6GnIHAeV5rK43FOas0JrIB7SYd8xkRRhJikvR+OxYSmpwJyHM5tEc25+srlFOekqx6MTPDMQwUMQ/wxx33ETla7Q3po5Ypjf8Z7g= |
| 2 | ELIZABETH | BLACKWELL | F | 1821-02-03 | E4G7QvnP21Q7Wp71rTE7uMWEkB03SrFuJ4rhLBPrAx4IS41dAYLWIRsPeGkzlgmVmYrCa3rrABZPvAsKnUAeE+FHpJhUPacdfDYsBQuGu5I= | VTY0gtsgUZoIxfoFXzwchDqFkofPZdylJlszvW8g1OoFXCrpCFzLLxk0SdNkEXTBdNof0lA1Dqk9NPIBOZcW8UfcQjGlb66Jo4TRvc9kXUo= | bOTIQg+gcOQ0NToyORzUJrf64oRMlHHRZBQPDueJzHBl1tIzq6l9Y/plH1N9K7Gy3FVSPsqNl+6SWpkgTWKinJKEQLt2wiCVbLgAw86Vc/U= |
| 3 | EDWARD | JENNER | M | 1749-05-17 | 4wGL1/yszKMjW7E3d2pk5fTcWLAKDp/4Dc03sIL0DG27Cu+QZUYYMf8pIyBwL/6ZMPhmr2X9QsOLTaq05ri09g6zB8HhCDuHUtRavdM0x1w= | 4j8xonF4Lcu8dpzcKwaw9UD1Lxuz5AcZv/fOnHwZhVifVDsWNEKVvyem3s4zqb8t6WeNSsC/rd6MYWKtZp5uvbq4pnqmC4gf1cwJa1E7dc8= | 7XIFi7mFqbvKK5NhZHSz6ZWIoL7ZyOP2QtHxSfhwwhThSnfe2deudM5hb+3RAc7r1Osb8wXd4mkqu9zlY1CUHc6HM2eah7TYCGHGTeLgRV0= |
| 4 | ALEXANDER | FLEMING | M | 1881-08-06 | kxwqjyikxC7WNxuX7cHCZp+Q7XEG5IYuVEshHi4RsyLgjM3lHHhi60Z3v5IpxKznFc+QNdea0S1vUyqyXDcbmzP597l91ZR4da7+cqA2Z4Q= | GD/IeZRqsRYWn/D8YfTw4FS+gT6bAM4oPvHlswcrc62n3KBH1ungFhTdHmYZXnOlsTVrFBluxcv5qUNCxDrxMgczN3StZZd+LS9YaSb+fsQ= | 00MN/EeN8M2ILTLLdgw0qNwjDfyBOROoKZydE05dXlPlv7Y+JVuoOpFdzlSu3KF4gTRhyw8B8qBWO7uwq5wAHS+poqtP/sHf4xqcCZ8WI4Y= |
Transcrypt Tokens As Sender¶
trans- (latin: across) -crypt (greek: hidden, secret)
The OPPRL specification also provides procedures for facilitating the safe transfer of data between trusted parties without a secure connection.
The process of Transcryption is the re-encrypting of data from one scheme to another without reversing the initial encryption to recover the PII. When the sender transcrypts their tokens, the token values of the records no longer match to the sender's data and records within the data being shared no longer match with eachother regardless of the underlying PII.
Notice that the first 2 records pertaining to the same subject (label = 1) no longer have identical tokens.
In [0]:
Copied!
from carduus.token import transcrypt_out
tokens_to_send = transcrypt_out(
tokens,
token_columns=("opprl_token_1", "opprl_token_2", "opprl_token_3"),
recipient_public_key=dbutils.secrets.getBytes("carduus", "MyPartnerPublicKey"),
private_key=dbutils.secrets.getBytes("carduus", "PrivateKey"),
).drop("first_name", "last_name", "birth_date")
display(tokens_to_send)
from carduus.token import transcrypt_out
tokens_to_send = transcrypt_out(
tokens,
token_columns=("opprl_token_1", "opprl_token_2", "opprl_token_3"),
recipient_public_key=dbutils.secrets.getBytes("carduus", "MyPartnerPublicKey"),
private_key=dbutils.secrets.getBytes("carduus", "PrivateKey"),
).drop("first_name", "last_name", "birth_date")
display(tokens_to_send)
| label | gender | opprl_token_1 | opprl_token_2 | opprl_token_3 |
|---|---|---|---|---|
| 1 | M | fUxhkLG+G9blbk/zH7TFlIQy23OjevfCmYlb1yMS85NvcumQLLpKiz7QcTaFsWE9CB80gvzFI6i4m4By+iGy8xDTCSaxry7BkWr5L3jn2T5mcf1ZpSOtlNJcWS9Ts5rVbigsO8EsWJSTn61o78VbOb8YZ/2p7AT63dUqqCBM7pabv9MH2I2QIzjzwgEZ0C9BmRmsXkkZ4JMstaJu+lIWZyyMFzBKe471X5rRo4HL0FxT+VSMdXz/IFeNSAImJCtBglEfCMZhfCMqU1MKG+BsqXK1mhujmm20neIwJMOqnIueG3pWKXtcn/btcaFUnwzTs793L3lZt57mrHmiJTLLVw== | U7uj6e8Y5qBanGmZsPvV36cA3zhoH/kWyTv/ffC+aRlrzjUyQe4jc9bJQ+UG9OsdOwcrcr1OI+EoovhqGBpJOCgWud4tWz9SPQMRh2Cf54qbG+ODYWCxKSTpPwJ7+SHPf+8iKebVEK7wT6iYaw6GS5OLqxYKL3hHcpZupV8bu5GZfzGpeqNl8XzwE1A/H+pwrusZziRbMgoRIaW49VnrKH3Z4R5GbH0zwUEZTSN3rguljNQgtG1RqL/njy/RtQo7wh7jusDA5GHvd7X0tCDCxN/VP6NkSPj5r/xLXvgcPQwLEbG1UxR7sUldFxzl0TnLIWJNmi3fJAAbSu10dP3pTQ== | jGnVejN4ODaSh+V0jA1rOrj4lpFXYprnZ+SFP63oFWNjmDudsRo1KIGTxSXi977j8OXM6HTFBUMS3PzVGvr+ixDx7VuEEJ/QfiTI5XLDlyim71sXn1YzDn0hSDFPHEF39vAL5M5oii6uCEsdEroTccfOgWQj6J+F/GAPwL7otxa5X7S7ZPJqMDTQ3E8VmLQSFImwG19SaTdxcFFwFJ4bfR0DaIIvb8/e/RrRT4oHSqcoZDIHo4jHS/7DaM+QwA9/hulErOfVUKPDqtpBgs88sL/MwHgXtkNo2xsK0mQ4hlVkETbMv4amgrnFtIWAVUx/ngT6jt1dAtbH0P8x3p0V0w== |
| 1 | M | gk0V1PNQ8fsPtqOm4tPEQY0M1Dic4o3l1oO/cS3XTHdujoKOmAR1sDfdvrGboNfR3hmmLCgSfQc+n7kxvbGKkMIRfc/DMcBCaYgr+3vVyE+N8d5kXd6hZE1WrCdWOgj1prxk/70Hk53n05kLQxppLney2NSgkBj6FwrJ9VXrbCm4zqTAwlwWdNBweX+7qRhuMbxUiXkChx52OqgPGdpIvMFNAJVtL/B11/3hIgh3soUKncKXEv8HChs18wJbJHrHLRRwcZS19DTctaJkKQxqqqMOlzYHAGP6lVczQGIaUh7m8Y0JCfYNpkg+3YATRf1EOOmAmgrYYuJLt4l/BHVKWg== | HsSe2Dj8OjRUFzWEaTBfd46mEY4kf/KBvNfwY+pe2btJ2ES6mTaXt4QS5bRw8HyMmazjUF8Hz0pBnZZGfO236mscDvv8TaK5cRrn5HROXBxVXdpb/fz8CumggXKobY33iuk8bD97NpA6R4fwTnphtQc8E9ofrW1tsecFHYJNFO4HefjlKaeS0x/dQ9bEU1iD/n7vtNZVBp5jNQuLktSjTer+tMACvdrDPLZFL5hOSa0nOuA32+g9GYQD5pZUJ6r4LM87IHXF+ggf/Q64/aeDzFVIGiznsqsKg/cUm9msrY/thcOvYXWjb1r1XND3d/z3zjInKcTZ75JgVwLoXQbVIQ== | dJDc+FiR8PZ9eco5WUBNUQ/Mnp3WRBmJOgvDUwNi09fA32Q5CvOQtfGaV7fqK0meFdP2OM7/XBAr4ZZhvp7UG/SHiZkOYTQxyKUgEAFtuEf9MpUamX5fA4M11h/bE95YPfabkuw6R52Ghxmg3yvQCM8Mv8JZjYJU416ko88+TjaMv8TKGM5mRxhpfkfOrdAgUENFVCLAi9Q95Glu26Y8EXuVqHAYfdILFe9XTpaM9BtZbfTPgFs7yPAtO/RqHVRGMGnuGwSNLhBViN/QV1uUys5tuU5yFT9MzM9VBifwHIpiGvl0l+j9fwZclergwuf3e4vGuFMnQZmDCuMAdzRsbw== |
| 2 | F | QXS7khXYHRoNrE5Oqpy04c9+PtrOrSrgGtzje8cMVDuBTH63gsJxqeHYKk1IXaiRcniFtDaQ4M4+lOuMlnPNhMihd5kxD4fMUarL5XffQtBFr8H3gQDRbSV0kxFPJHVqtXBWO2pNWe6DN/cYJR7XtGSwY0dyWXWTRhkt6YCc5DP6uah3OUbKQEzCgONYXdHDyn7qsdOprz6EU15e2xSwfrnfH8ysYfAAHU/gQKJhZXmXtPqt5vzkXSm6H2A+4n/+AY/Nd/Qz9kl4zrqzWsctyzqDHLtw01O4w60bNV4lekYLkCBOOManC5wHwAni3PxtE5XzA4HJqHXKiBCAWAzpLg== | ef3MO6eLNr8wHv9CMYXSbbEj8e+CUwJMbVVQql0Hsj/3V99SXflIP/Bec7iOaeX/nymKyAHaDkDjimp8EZqeboPa9p8BVViW0n1Tor6XRW7WRsmHjwLcKyJ1IGgRk08rO6PdLTUcHRAQXtsvlJuFq3EabqZk3ZliZcmljpVij3MKsnQ8VVYKZYGz17yUzgUIOXMT1C3Ouw0SjSgViI5vEQj0nTY9NrZyIN5FS4v4tf0x/Lm71uXpiiVULGEm1eHpV+I1UiP0J2O/fIUd50LoDIfC0G8dZXSoak55Rc1HI8F2Y6dyavuxi2t/k+bF8A6fFuy7vk9WEsKPiyDZ8AcN6A== | WD9jVu0cbXhq4p7NQbI2vhFCdvU4GI/kHUjTmXzp7OLY7BvEbTNxlDvrEbeIkoxm8y/fcygVk3ioMMhZ81pFy+MxE2XlfsjWvVCers2Ltw+ua2dTS14FKRp8MYiFj7516rYt81hPINBK8/DzW9jtXqUdDWdHiivZDUBz8xPK+hmUuPdm8ppWPNelHS6autmjb1jBB6bEwKK38MEEkZRbE6Uj/78usd9f7YV2+HcTRy2jg2V6LcdxW8i0fNb5cJAKzKCabGt6r9WLISz8VYKJPNFMgAegpnywaPqAoBgBqFaqp2F8q0g/UTp3uxrQjKEtSVJ9i9ggVuhAK+Gwcfs1XA== |
| 3 | M | Hpp7B0EB4UvIxDYRbRVHXnhZL2ZgLNh9ALK7VsoQX276BsC4CfqwC0BLchSnHNEAj/6YYT3qsqg4AGbab0RnPfAjmBj8i8pBmndZK2kicyUbVKhzXABO5kEjBVPCMz/7g6fIysLHczHg3kd3cibyAyQTKDkYyoVhnwxG9Izj0aKBi5HzqDAkKopDwz/I228fEggoN39Pd8pbjdx2+a0BEFvJeI7m9twRsd26GxBLrXuVH6EGAVu/vDUEge2f8gCT9CaFCQq5JbyLf+uqxDBU2/GHT0IcEENtHpFCFNP3JHtnOPbICUPaZhpkc5u0TEz7O41oklslvQ/w8fEh7aNnfg== | T/DoAbgx27LunmbwlY3WcTbAwYJRQeOEx6E0KSdvzM+n5dhMB39uWAdu9qilR4xnqw2+Vbt1dP+/M4q7tTOo00w990IVdHfaPAOTU/53YmjgkKqT/VzG9MZYIx8tqr3MTSmvwOtzz4yXKF3mtCg+xWI09zP96WYktLsbZKenZwNsHXtyAa3Zl5/FAas4Q8CTFg3F8xN+joKZCyWZVnnTRTG5SCyJW7M1dlr+nMMm9Awee6NvpgYBXHKfLWwk+aMJxpcR/311RUvQUEMuc52L7hsAHF+t8r81mIysGIuuKaCAR29KtYKNmYA2dxI0hJfs1lvLOmAVfxm6lSifAsgg0w== | Ya2uz/a8dEV7D4X95pd/qfUEmkb8nMIEsTzCCdwHm6OkkNO++O0bjAlCuCeMcNVxAEBw6RtW3QGZH87/j3EWCzyUEGSS/ZRQbfcDt/BSx6YCEjA/ZfEHrpKWw/fSoLq8kO8X3CNzkB3T40rCtbp3c+376Qg0EswgquLvWxhLgBeRnDuM5jh/PKuvxPz3GBIP6JKVKy3L3tBdxv+tw2RqlUmbIh2/El3xT6K6jp0fOtcdI61tKY9B2mYoncGF9jyzZo8Mg/dr9OLbswhV6DLYeUbnJN8osT54EQ5dtdwZ6e62BsSSzzBQkpgrAA8IHMaiiPKhtMJJvr88HjHMN8zbrA== |
| 4 | M | a7ajv6ohrH3dZw12jt5CkSyOKhCqpvb0RGP4t8c58BWm0VrjbU2nzOwp1tXtJxOnH7uA1v7kyGbyv4sRY4X7rP3xzVVcgISygpdcfD+aF8ZdwTeHYHouwbgw8S6Ms0lIHNKKgZI74Wxj/m/WU6qt1ZVee8RYpb4a0O/5DbUBIC69Mg4QNHB0yjPfBXnf+b9ZdjfX58uQWEVAvLrE+hxEX73TEsOa5HVr+49FRdMq9zJy4YC8tjoOheRrhsb6oArIzZFqL9k2sc2SigJsdqWJMi8dAZFu2Xiw1nLPPWb/7WlyEcTjtu83BLfxh8Mif5xmh75FuoplA+6CRM863l7Zzw== | K8wHTWPPDLBouAH0brBmbPLBMQyUWzPEB1ifobca6lWIwIoy/H0zPGvStngqOTfNcHkO00TkU691gGZbKP5PluCz11onslBJXYhATxx03WpbWXCai/uyb9fmXRwmWxFXNhrZ9gaVhPDhnQ8JuQT35bcP4XckcmNNOlz2z5KG11d6icJ4ExkdZWFDa/9KZqOekYWmbkZJB4Z6kFBfn1RmnsiLWIG+iBV4vcG9XDIu3xECt4cg69jog4nzRKXQXZsLsDAhCkZ2dyGTOXERlZp4H3+gklwuA7rUWG3/EpBFlzsbhnx4J7E40CGl3Rlq6PAu7oiMNoODsCFqyuRhIC2RwA== | dk96GXnw9B5qaDHsOK90Dhg0ErenPL+9zY34ikYk+clTFFujgG9SJnKIhxYJQ1vsH5ExntoeY5BJdt5keUFZm4jN6XPI981p4eNdgtFPRdFddqVFBd/OotfDhoW9S9WtEKPKvKAbJZBmYhVXLvxKS95LkYhWeklIQQ4WxU4mEgVmgKQUUy115Wxy3N4jwFErJocrCnHPmfv4d18a0ch848ie1LXp3HnfRKaIWwNkZSxeDrofjHJxsGa3ETZcTPMHfYCdvkfC62mZKXBbVgZz2CPoD0D1iCN6wUeVHA3FpKg87mD30J5jVCoa93oq+LEBgMqgsJqrpxCdSO8wdT7Aaw== |
Transcrypt Tokens As Recipient¶
When receiving the data with transcrypted tokens, the recepiant must call a function to re-encrypt the tokens again so that they match with the rest of the tokens across their data assets.
For this demonstration, we will load the encryption key our hypothetical recipient and provess the transcrypted tokens we created above.
Notice that the first 2 records pertaining to the same subject (label = 1) have identical tokens again, but do these tokens are not the same as the orgional tokens because they are encrypted with the scheme for the recipient.
In [0]:
Copied!
from carduus.token import transcrypt_in
tokens2 = transcrypt_in(
tokens_to_send,
token_columns=("opprl_token_1", "opprl_token_2", "opprl_token_3"),
private_key=dbutils.secrets.getBytes("carduus", "OrgBPrivateKey"),
)
display(tokens2)
from carduus.token import transcrypt_in
tokens2 = transcrypt_in(
tokens_to_send,
token_columns=("opprl_token_1", "opprl_token_2", "opprl_token_3"),
private_key=dbutils.secrets.getBytes("carduus", "OrgBPrivateKey"),
)
display(tokens2)
| label | gender | opprl_token_1 | opprl_token_2 | opprl_token_3 |
|---|---|---|---|---|
| 1 | M | n6YNZapNFyiP3rPO6JmlNZnxfz4PL9lfn3IOoERk9cSLe2w91EDdKHL+nNgZ60SzeRQzcM3nHDRQevzOfkFLf4dJTJg+8uLtcqHa3sHn0D4= | gcbO8zyzdiMsWiWrChcskBcPHJ4z04CmZ4DrY+V20n6XaSpq8mJQ41um85gwSEcoOdIYE2ucFPn9mf3QKmNnBrRTYoA1q9KkSWK/Z6etbMg= | AaGOVbbebDiOSq/lMVORAyeEHf8RcfrvPF7SCNei1wcSK7IJph22uxOPnfmmNNpAwzo9Q6Tr8XYOXwxgOYY7JG6NcX9O/1qVSGvEq3wb17E= |
| 1 | M | n6YNZapNFyiP3rPO6JmlNZnxfz4PL9lfn3IOoERk9cSLe2w91EDdKHL+nNgZ60SzeRQzcM3nHDRQevzOfkFLf4dJTJg+8uLtcqHa3sHn0D4= | gcbO8zyzdiMsWiWrChcskBcPHJ4z04CmZ4DrY+V20n6XaSpq8mJQ41um85gwSEcoOdIYE2ucFPn9mf3QKmNnBrRTYoA1q9KkSWK/Z6etbMg= | AaGOVbbebDiOSq/lMVORAyeEHf8RcfrvPF7SCNei1wcSK7IJph22uxOPnfmmNNpAwzo9Q6Tr8XYOXwxgOYY7JG6NcX9O/1qVSGvEq3wb17E= |
| 2 | F | jSXy3EODCctLC6bYmp4PLwTyvIlruiSHVJG+PzkrffKsypBII409DuMDq+20mCoQIOnqZXqChkUNhrqiKPQX+E3wfFQWiJSEPk/HlHYKH7U= | w1XiSj7vz4R68DpkQxMqSfFikQvylx3ZPrOoBQWBH3CypDgShSmw+6tC5Ows+hKVZkan+13letcC4u8ifO612dlg63hYZ/ONfTSGevD14ao= | 5Fk8udPfGm44jIDkFbhRJYIQnG4jjgpMhygEX7eHacpy1fCnuOMJB/5IP87570eISwYWqsRSuDna8iRqywrj8KTiVbMoY7p8Mg7Je3uFGKY= |
| 3 | M | 5etiSY9RzaVmpefeRGR0ajR08O6aez6ZrJUoAKrfIrNzOPHLAQqi4tXIp+4kvIhp4YygsXjleWirtfs8hbl0fwFWeGQ4vUBBsXUY0jG8MlU= | WJ340Z+/qZy/5yTvABqJMYgrkTzZtXdHmKW/OduRiIpidBgVF8iijUfi/+M8nOKnUkgFbsCrqFtI57XP5GNuAvFmptyRS6EqPbOupF8PcNE= | xnR2qS0zGdfBZrmBomMJezAPv/UYCQdpW0nFV7KiZotpYG5Q/d9EK5Ew27SaFP2dJNNVCwCEXCV+jM+RBXIuMnxbVWvy8FjPODx3UR5Qrrg= |
| 4 | M | tfhWByB2vs7mTwVf1xQZ7n4KVf9U9JqdeINK6DJtMj/7msCQVAljCH+X58pbdjLfMDZlrMHZtiVjpl5i9nfrcj/iou5IClUnycIhupaT8JM= | wBgFLcQj/QwyWe7YXrnHvNXoMQvNOmaBFi6q0gMbVSuO1f8xQk8fXoBN18/SH50amlmLs+4GV1eYatY+rvVEFVb4RJMSds91ZH/yjyLIkTw= | /OfhKKqUTgAS+vABrSVG9sW0CrpiJmHDkWQ9fTOBAKASwsY9IDnBufIoKSs3SXjr24rYBvdFWMLZWAGfVukz91/yy39MsgeX7MJERxHigAQ= |
In [0]:
Copied!