I'm trying to pull in google service account details as an airflow variable and write this to file when running a KubernetesPodOperator. I'm currently trying to test this in pylance with a launch.json file as:
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: Pipeline",
"type": "python",
"request": "launch",
"program": "src",
"console": "integratedTerminal",
"args": [
"--assets_path", "s3://test/models/",
"--folder_source", "s3://test/data/ner/qle_v1_with_postags/20210118_2100/",
"--folder_destination", "/tmp/test/new",
"--gcs_folder_destination", "gs://test/tmp/test/gcs",
"--aws_region", "us-west-2",
"--s3_endpoint", "s3.us-west-2.amazonaws.com",
"--gcs_credentials", """"type": "service_account",
"project_id": "test",
"private_key_id": "b618cd07af66a9d57786af2575a23a3821728297",
"private_key": "-----BEGIN PRIVATE KEY-----\nvsdjkfsdkjfbsajfklsdabfjubskfllbsdlafgbsafgjlsdabfjsadbfsjhda\njhdsgfhjegfsdfvgbdshjfgdshfgsdajhfsadkfh\nfgjkfdhgjkdfshgjkdfgl/dfkjsdhftkrshfbsjdkdffdnkljgfhdjfkghhdfkjlfghdskjsnhdfklhskjlfh/dfbhjsdfdgbhjsafghsdajkfgvshadjgfdggfhdfk\ndES0n7MPdwOW2sgXUKqAOGxZUiZvqGaY+rRBqmhIMYA1No4uou+nB1pyvR/tbGUn\nuPi6mnAeisUy6PKpbK/0m1rOdjta+xs2D3KSodw8FvZTZO8qhzP0k3tkYB/LMNdh\n079pZOV7AgMBAAECggEAFD7sMomWfpnmp2R1RKXe4fy4ISkFRLnyspMQhvQp/vQt\nb5TXmBIdXaA3m8kSZ/9ILHfMRPPfm06uvl5HDRAN6idQ+Jwez43ULJoa06wQizUS\nV\niYWzyLqVjBm/ztnhsPbazT9jz+CONsVgO2ViMm4BvBLv600JwivwFKxftJocQ5bi\ncLw3QBCdFgU64yfEdpYOCsNb\n-----END PRIVATE KEY-----\n",
"client_email": "ml-storage@test.iam.gserviceaccount.com",
"client_id": "103551600936651887697",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/ml-storage%40test.iam.gserviceaccount.com"
}"""
]
}
]
}
Unfortunately i can't seem to convert the gcs_credentials into a string for testing.
Aucun commentaire:
Enregistrer un commentaire