Our users have been able to upload their Gmail email and calendar metadata through

But there is also an alternative, if you are an Admin in a Google Business Suite you can collect all this data on-behalf of your employees.

Create a service account and allow this service account access to impersonate users.

To generate service-account credentials, or to view the public credentials that you've already generated, do the following:

  1. Open the Service accounts page. If prompted, create a new project, we have noticed some permissions errors when using an existing project.
  2. Click Create service account
  3. In the Create service account window, type a name for the service account, and select Furnish a new private key. You have to grant G Suite domain-wide authority to the service account, to do so, select Enable G Suite Domain-wide Delegation. You do not need to select or specific role. Then click Create.

Your new public/private key pair is generated and downloaded to your machine; it serves as the only copy of this key. You are responsible for storing it securely.
Take note of the service account's email address and store the service account's JSON private key file in a location accessible to your application. Your application needs them to make authorized API calls.

Next, an administrator of the G Suite domain must complete the following steps:

  1. Go to your G Suite domain’s Admin console.
  2. Select Security from the list of controls. If you don't see Security listed, select More controls from the gray bar at the bottom of the page, then select Security from the list of controls. If you can't see the controls, make sure you're signed in as an administrator for the domain.
  3. Select Show more and then Advanced settings from the list of options.
  4. Select Manage API client access in the Authentication section.
  5. In the Client Name field enter the service account's Client ID. You can find your service account's client ID in the Service accounts page.
  6. In the One or More API Scopes field enter the following list of scopes, separated by a comma:
    https://www.googleapis.com/auth/gmail.readonly, https://www.googleapis.com/auth/calendar.readonly

    (more info about these scopes can be found here: https://developers.google.com/identity/protocols/googlescopes)
  7. Click Authorize.

Your application now has the authority to make API calls as users in your domain (to "impersonate" users). When you prepare to make authorized API calls, you specify the user to impersonate.

We have noticed some strange permissions errors when trying to update the scopes after making a mistake initially. Our recommendation is to create a new project, with a new client service account, if you made any mistakes in the above steps.

Errors like: "unauthorized_client: Client is unauthorized to retrieve access tokens using this method", while having the scopes correctly set could only be fixed by creating a new project and service account.

Example: email collection code

##################################################
#!/usr/bin/env python
# (c) 2018, Zugata Inc
##################################################

import httplib2
import sys
from apiclient import discovery
import time
import dateutil.parser as parser
from datetime import datetime
import datetime

SCOPES = ['https://www.googleapis.com/auth/gmail.readonly', 'https://www.googleapis.com/auth/calendar.readonly']

SECRET_FILE = '<REPLACE WITH YOUR JSON CREDENTIALS>'

CLIENT_EMAIL = '<REPLACE WITH THE EMAIL OF THE USER>'

DOMAIN_NAME = '<REPLACE WITH YOUR DOMAIN>'

def get_credentials():
  from oauth2client.service_account import ServiceAccountCredentials
  credentials = ServiceAccountCredentials.from_json_keyfile_name(SECRET_FILE, scopes=SCOPES)
  credentials = credentials.create_delegated(CLIENT_EMAIL)
  return credentials

def get_message(service, user_id, msg_id):
  try:
    temp_dict = { }
    message = service.users().messages().get(userId=user_id, id=msg_id, format='metadata').execute()
    payld = message['payload']
    headr = payld['headers']
    for one in headr: # getting the date
      if one['name'] == 'Date':
        msg_date = one['value']
        date_parse = (parser.parse(msg_date))
        temp_dict['Date'] = datetime.datetime.fromtimestamp(date_parse.timestamp()).strftime('%Y-%m-%d %H:%M:%S')
      else:
        pass
    for two in headr: # getting the Sender
      if two['name'] == 'From':
        msg_from = two['value']
        temp_dict['From'] = msg_from
      else:
        pass
    for three in headr: # getting the Recipient
      if three['name'] == 'To':
        msg_to = three['value']
        temp_dict['To'] = msg_to
      else:
        pass
    for four in headr: # getting the CC
      if four['name'] == 'Cc':
        msg_cc = four['value']
        temp_dict['Cc'] = msg_cc
      else:
        pass
    for five in headr: # getting the Message-Id
      if five['name'] == 'Message-Id':
        msg_id = five['value']
        temp_dict['Message-Id'] = msg_id
      else:
        pass
    for six in headr: # getting the In-Reply-To
      if six['name'] == 'In-Reply-To':
        msg_reply_to = six['value']
        temp_dict['In-Reply-To'] = msg_reply_to
      else:
        pass
    return temp_dict
  except:
    print('Exception in get message: ', sys.exc_info()[0])
    raise

def list_messages_matching_query(service, user_id, query=''):
  try:
    response = service.users().messages().list(userId=user_id,
                                               q=query, maxResults=100).execute()
    messages = []
    if 'messages' in response:
      messages.extend(response['messages'])
    while 'nextPageToken' in response:
      page_token = response['nextPageToken']
      response = service.users().messages().list(userId=user_id, q=query,
                                         pageToken=page_token, maxResults=100).execute()
      messages.extend(response['messages'])
    return messages
  except:
    print('Exception in list messages: ', sys.exc_info()[0])
    raise

def main():
  try:
      credentials = get_credentials()
      http = credentials.authorize(httplib2.Http())
      service = discovery.build('gmail', 'v1', http=http)
      response = service.users().messages().list(userId=CLIENT_EMAIL).execute()
      q = 'newer_than:30d' + ' from:' + DOMAIN_NAME
      messages = list_messages_matching_query(service, CLIENT_EMAIL, q)
      for message in messages:
        print(get_message(service, CLIENT_EMAIL, message["id"]))    
  except:
    print('Exception: ', sys.exc_info()[0])

if __name__ == '__main__':
    main()

Did this answer your question?