V2/university/T01006-Introduction-to-transaction-configuration/T01006-Ensuring-different-transaction-codes-have-the-same-economic-impact.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "identical-fundamentals",
   "metadata": {},
   "source": [
    "# Ensuring different transaction codes have the same economic impact\n",
    "\n",
    "In this training module we'll see how to use LUSID to perform the following task:\n",
    "\n",
    "**<div align=\"center\">As a data controller, I load transactions from two providers that use a different transaction code to signal the same economic activity. I want to ensure that LUSID applies a uniform economic impact when generating holdings.</div>**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "progressive-origin",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set up LUSID\n",
    "import os\n",
    "import pandas as pd\n",
    "import json\n",
    "import uuid\n",
    "from IPython.core.display import HTML\n",
    "from datetime import datetime, timedelta\n",
    "import logging\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "\n",
    "import lusid as lu\n",
    "import lusid.api as la\n",
    "import lusid.models as lm\n",
    "\n",
    "from lusidjam import RefreshingToken\n",
    "from lusid.extensions import (\n",
    "    SyncApiClientFactory,\n",
    "    ArgsConfigurationLoader,\n",
    "    EnvironmentVariablesConfigurationLoader,\n",
    "    SecretsFileConfigurationLoader\n",
    ")\n",
    "from finbourne_sdk_utils.pandas_utils.lusid_pandas import lusid_response_to_data_frame\n",
    "from finbourne_sdk_utils.jupyter_tools import StopExecution\n",
    "from finbourne_sdk_utils.lpt.lpt import to_date\n",
    "\n",
    "# Set pandas display options\n",
    "pd.set_option(\"display.max_columns\", None)\n",
    "pd.set_option(\"display.max_rows\", None)\n",
    "pd.options.display.float_format = \"{:,.2f}\".format\n",
    "\n",
    "# Authenticate to SDK\n",
    "# Run the Notebook in Jupyterhub for your LUSID domain and authenticate automatically\n",
    "secrets_path = os.getenv(\"FBN_SECRETS_PATH\")\n",
    "# Run the Notebook locally using a secrets file (see https://support.lusid.com/knowledgebase/article/KA-01663)\n",
    "if secrets_path is None:\n",
    "    secrets_path = os.path.join(os.path.dirname(os.getcwd()), \"secrets.json\")\n",
    "\n",
    "# Initiate an API Factory which is the client side object for interacting with LUSID APIs\n",
    "config_loaders=[\n",
    "    ArgsConfigurationLoader(access_token = RefreshingToken(), app_name = \"LusidJupyterNotebook\"),\n",
    "    EnvironmentVariablesConfigurationLoader(),\n",
    "    SecretsFileConfigurationLoader(secrets_path)]\n",
    "api_factory = SyncApiClientFactory(config_loaders=config_loaders)\n",
    "    \n",
    "# Confirm success by printing SDK version\n",
    "api_status = pd.DataFrame(api_factory.build(lu.ApplicationMetadataApi).get_lusid_versions().to_dict())\n",
    "display(api_status)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "major-provision",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a scope and code to segregate data in this module from other modules\n",
    "module_scope = \"FBNUniversity\"\n",
    "module_code = \"Module-6-1\"\n",
    "print(f\"'{module_scope}/{module_code}' scope and code created.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "responsible-booking",
   "metadata": {},
   "source": [
    "## 1. Examining the source file\n",
    "\n",
    "Imagine we load transactions into LUSID from both Societe Generale and Deutsche Bank:\n",
    "\n",
    "* SG uses a transaction code of `Acheter` to signal an equity purchase.\n",
    "* DB uses a transaction code of `Kaufen` to signal an equity purchase.\n",
    "\n",
    "We want to retain these codes in the imported data but ensure LUSID applies the same economic impact to every transaction when generating holdings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "independent-first",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read transactions from amalgamated stream into pandas dataframe\n",
    "sg_db_df = pd.read_csv(\"data/different-codes-sg-db.csv\", keep_default_na = False)\n",
    "sg_db_df.index += 1\n",
    "display(sg_db_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "alone-fountain",
   "metadata": {},
   "source": [
    "## 2. Ensuring instruments are mastered correctly\n",
    "\n",
    "It's possible the equity instrument in our transaction source file is already mastered in LUSID as part of the demonstration data, but for the avoidance of doubt we'll master it separately in a segregated custom instrument scope.\n",
    "\n",
    "Note the `GBP` currency instrument, as per all currencies, is mastered out-of-the-box in the `default` instrument scope."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sustained-director",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain the LUSID Instruments API\n",
    "instruments_api = api_factory.build(la.InstrumentsApi)\n",
    "\n",
    "# Create a dictionary of instrument definitions for each equity in the transaction source file (ie. BP)\n",
    "definitions = {\n",
    "    security[\"instrument\"]: lm.InstrumentDefinition(\n",
    "            name = security[\"instrument\"],\n",
    "            identifiers = {\n",
    "                \"Figi\": lm.InstrumentIdValue(value = security[\"figi\"]),\n",
    "            },\n",
    "            definition = lm.Equity(\n",
    "                instrument_type = \"Equity\",\n",
    "                dom_ccy = security[\"currency\"],\n",
    "                identifiers = {}\n",
    "            )\n",
    "        )\n",
    "    for index, security in sg_db_df.iterrows() if security[\"asset\"] == \"Equity\"\n",
    "}\n",
    "\n",
    "# Upsert instruments to LUSID\n",
    "upsert_instruments_response = instruments_api.upsert_instruments(\n",
    "    request_body = definitions,\n",
    "    # Master the instruments in a custom scope\n",
    "    scope = f\"{module_scope}{module_code}\",\n",
    ")\n",
    "\n",
    "# Transform API response to a dataframe and show internally-generated unique LUID for each mastered instrument\n",
    "upsert_instruments_response_df = lusid_response_to_data_frame(list(upsert_instruments_response.values.values()))\n",
    "display(upsert_instruments_response_df[[\"name\", \"lusidInstrumentId\"]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "stopped-texture",
   "metadata": {},
   "source": [
    "## 3. Creating a suitable portfolio \n",
    "We register the custom instrument scope using the `instrument_scopes` field to ensure transactions upserted to the portfolio resolve first against instruments mastered in the custom scope, before falling back to the `default` scope."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "swiss-stream",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain the LUSID Transaction Portfolio API\n",
    "transaction_portfolios_api = api_factory.build(la.TransactionPortfoliosApi)\n",
    "\n",
    "# Create portfolio definition\n",
    "portfolio_definition=lm.CreateTransactionPortfolioRequest(\n",
    "    display_name=\"Training module 6.1 - Different transaction codes, same impact\",\n",
    "    code = module_code,\n",
    "    base_currency = \"GBP\",\n",
    "    # Must be before first transaction recorded\n",
    "    created=\"2022-01-01T00:00:00Z\",\n",
    "    # Attempt to resolve transactions to instruments in the custom scope before falling back to the default scope\n",
    "    instrument_scopes = [f\"{module_scope}{module_code}\"],\n",
    ")\n",
    "\n",
    "# Upsert portfolio to LUSID, making sure it's not already there\n",
    "try:\n",
    "    create_portfolio_response=transaction_portfolios_api.create_portfolio(\n",
    "        scope = module_scope,\n",
    "        create_transaction_portfolio_request = portfolio_definition\n",
    "    )\n",
    "    # Confirm success\n",
    "    print(f\"Portfolio with display name '{create_portfolio_response.display_name}' created effective {str(create_portfolio_response.created)}\")\n",
    "except lu.ApiException as e:\n",
    "    if json.loads(e.body)[\"name\"] == \"PortfolioWithIdAlreadyExists\":\n",
    "            logging.info(json.loads(e.body)[\"title\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "curious-crest",
   "metadata": {},
   "source": [
    "## 4. Configuring the built-in Buy transaction type\n",
    "\n",
    "The built-in `Buy` transaction type is designed to increase an instrument holding by the number of units purchased, and decrease a cash holding by the total consideration.\n",
    "\n",
    "We want `Acheter` and `Kaufen` to have the same economic impact.\n",
    "\n",
    "### 4.1 Retrieving the definition of Buy to obtain the correct settings\n",
    "\n",
    "We can call the `GetTransactionType` API to retrieve the definition of `Buy` before appending an alias for each of `Acheter` and `Kaufen`, to ensure we retain all the other settings correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "floating-southwest",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain the LUSID Transaction Type Configuration API\n",
    "transaction_config_api = api_factory.build(la.TransactionConfigurationApi)\n",
    "\n",
    "# Get existing transaction type definition    \n",
    "try:\n",
    "    response = transaction_config_api.get_transaction_type(\n",
    "        # Specify the source of the transaction type\n",
    "        source = \"default\",\n",
    "        # Specify the primary alias name (in this case, the name of the only alias in the transaction type)\n",
    "        type = \"Buy\",\n",
    "    )\n",
    "    display(response)\n",
    "except lu.ApiException as e:\n",
    "    if json.loads(e.body)[\"name\"] == \"TransactionTypeNotFound\":\n",
    "        logging.info(\"Transaction type not found\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cross-hospital",
   "metadata": {},
   "source": [
    "### 4.2 Modifying the definition of Buy to append new aliases\n",
    "\n",
    "We can now create a new definition for `Buy` that appends aliases for `Acheter` and `Kaufen` but retains all the other settings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "southwest-grave",
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_type_definition = lm.TransactionTypeRequest(\n",
    "    aliases = [\n",
    "        # Recreate the built-in Buy alias\n",
    "        lm.TransactionTypeAlias(\n",
    "            type = \"Buy\",\n",
    "            description = \"Purchase\",\n",
    "            transaction_class = \"Basic\",\n",
    "            transaction_roles = \"LongLonger\"\n",
    "        ),\n",
    "        # Append a new alias for Acheter\n",
    "        lm.TransactionTypeAlias(\n",
    "            type = \"Acheter\",\n",
    "            description = \"Purchase from Societe Generale\",\n",
    "            transaction_class = \"Basic\",\n",
    "            transaction_roles = \"LongLonger\"\n",
    "        ),\n",
    "        # Append a new alias for Kaufen\n",
    "        lm.TransactionTypeAlias(\n",
    "            type = \"Kaufen\",\n",
    "            description = \"Purchase from Deutsche Bank\",\n",
    "            transaction_class = \"Basic\",\n",
    "            transaction_roles = \"LongLonger\"\n",
    "        )\n",
    "    ],\n",
    "    # Recreate the built-in Buy movements\n",
    "    movements = [\n",
    "        lm.TransactionTypeMovement(\n",
    "            movement_types = \"StockMovement\",\n",
    "            side = \"Side1\",\n",
    "            direction = 1,\n",
    "        ),\n",
    "        lm.TransactionTypeMovement(\n",
    "            movement_types = \"CashCommitment\",\n",
    "            side = \"Side2\",\n",
    "            direction = -1,\n",
    "        ),\n",
    "    ]\n",
    ")\n",
    "\n",
    "# Replace transaction type in LUSID    \n",
    "try:\n",
    "    response = transaction_config_api.set_transaction_type(\n",
    "        # Specify the source (data provider) of the transaction type (Buy is in the 'default' source)\n",
    "        source = \"default\",\n",
    "        # Specify the primary alias name\n",
    "        type = \"Buy\",\n",
    "        transaction_type_request = transaction_type_definition\n",
    "    )\n",
    "    # Confirm by retrieving the newly-modified transaction type\n",
    "    display(transaction_config_api.get_transaction_type(source = \"default\", type = \"Buy\"))\n",
    "except lu.ApiException as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "infrared-settlement",
   "metadata": {},
   "source": [
    "## 5. Loading transactions into the portfolio\n",
    "\n",
    "We can now upsert transactions into LUSID, mapping the **txn_code** column in the source file to the `type` field, safe in the knowledge that the `Acheter` and `Kaufen` transaction types will have the same economic impact as `Buy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "restricted-interview",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create list of transactions to upsert\n",
    "transactions = []\n",
    "\n",
    "# For each row in dataframe\n",
    "for index, txn in sg_db_df.iterrows():\n",
    "\n",
    "    # Set variables based on whether instrument is cash or not\n",
    "    if txn[\"asset\"] == \"Cash\":\n",
    "        # Set the unique instrument identifier to be a currency\n",
    "        identifiers = {\"Instrument/default/Currency\": txn[\"currency\"]}\n",
    "    else:\n",
    "        # Set the unique instrument identifier to be a FIGI\n",
    "        identifiers = {\"Instrument/default/Figi\": txn[\"figi\"]}\n",
    "\n",
    "    transactions.append(\n",
    "        lm.TransactionRequest(\n",
    "            transaction_id = str(txn[\"txn_id\"]),\n",
    "            # Set the transaction type to the transaction code from the data provider\n",
    "            type = txn[\"txn_code\"],\n",
    "            instrument_identifiers = identifiers,\n",
    "            transaction_date = txn[\"trade_date\"],\n",
    "            settlement_date = txn[\"settle_date\"],\n",
    "            units = txn[\"units\"],\n",
    "            transaction_price = lm.TransactionPrice(price = txn[\"price\"], type=\"Price\"),\n",
    "            total_consideration = lm.CurrencyAndAmount(\n",
    "                # Calculate cost on-the-fly\n",
    "                amount = txn[\"units\"] * txn[\"price\"],\n",
    "                currency = txn[\"currency\"]\n",
    "            ),\n",
    "        )\n",
    "    )\n",
    "\n",
    "# Upsert transactions to LUSID\n",
    "upsert_transactions_response = transaction_portfolios_api.upsert_transactions(\n",
    "    scope = module_scope, \n",
    "    code = module_code, \n",
    "    transaction_request = transactions\n",
    ")\n",
    "\n",
    "display(f\"Transactions loaded at {str(upsert_transactions_response.version.as_at_date)}\")\n",
    "display(upsert_transactions_response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "south-parameter",
   "metadata": {},
   "source": [
    "## 6. Generating holdings\n",
    "\n",
    "### 6.1 On the trade date\n",
    "We can now call the `GetHoldings` API for end-of-day on the trade date (6 June 2022) to see the interim impact of our transaction types on holdings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "hollywood-literature",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get holdings for portfolio effective EOD 6 June 2022\n",
    "get_holdings_response=transaction_portfolios_api.get_holdings(\n",
    "    scope = module_scope, \n",
    "    code = module_code,\n",
    "    # Specify an explicit date\n",
    "    effective_at = \"2022-06-06T17:00:00Z\",\n",
    "    # Decorate the instrument name property onto holdings to make the API response more intuitive\n",
    "    property_keys=[\"Instrument/default/Name\"],\n",
    ")\n",
    "\n",
    "# Transform API response to a pandas dataframe and show it\n",
    "get_holdings_response_df=lusid_response_to_data_frame(get_holdings_response, rename_properties=True)\n",
    "# Drop some noisy columns\n",
    "# get_holdings_response_df.drop(columns=[\n",
    "#     \"sub_holding_keys\", \"cost_portfolio_ccy.currency\", \"currency\", \"SourcePortfolioId(default-Properties)\", \"SourcePortfolioScope(default-Properties)\" ], inplace=True)\n",
    "display(get_holdings_response_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "designed-evidence",
   "metadata": {},
   "source": [
    "### 6.2 Post-settlement date\n",
    "\n",
    "If we call `GetHoldings` API without an explicit date, LUSID uses today's date so we can see the impact on holdings after the settlement date."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "breathing-concrete",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get holdings for portfolio effective today\n",
    "get_holdings_response=transaction_portfolios_api.get_holdings(\n",
    "    scope = module_scope, \n",
    "    code = module_code,\n",
    "    # Decorate the instrument name property onto holdings to make the API response more intuitive\n",
    "    property_keys=[\"Instrument/default/Name\"],\n",
    ")\n",
    "\n",
    "# Transform API response to a pandas dataframe and show it\n",
    "get_holdings_response_df=lusid_response_to_data_frame(get_holdings_response, rename_properties=True)\n",
    "# Drop some noisy columns\n",
    "# get_holdings_response_df.drop(columns=[\n",
    "#     \"sub_holding_keys\", \"cost_portfolio_ccy.currency\", \"currency\", \"SourcePortfolioId(default-Properties)\", \"SourcePortfolioScope(default-Properties)\"], inplace=True)\n",
    "display(get_holdings_response_df)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}