-
Notifications
You must be signed in to change notification settings - Fork 28
Deploy Prajna
This document describes the basics on Prajna deployment.
The essential of Prajna deployment to a cluster (or just a set of machines) is to copy and launch the Prajna client to each properly provisioned machine.
A Prajna client is going to use
- a port for accepting job requests
- a range of ports for jobs
So the most basic requirement for a machine is to open these ports. Thereafter, the Prajna client can be launched using these ports. Of course, these machines need to be accessible from the machine where user runs Prajna applications.
The client is called "PrajnaClient.exe"
- For user of the Nuget package, when the Prajna package is installed, the client and its dependencies are located in subfolder "tools\Client"
- For user of the GitHub source code, after the source tree is built, the client and its dependencies are located in subfolder "bin\ReleaseX64\Client" or "bin\DebugX64\Client"
To launch the client, use the following arguments:
-port 1005 -jobports 1250-1300
An extra argument can be provided if request authentication is desired:
-pwd somepasswd
Depending on the environment and user preference, the way to provision, deploy, and launch the client can vary. A section below shows an example of using powershell scripts to deploy to a set of domain joined Windows machines.
Once the client is deployed to a set of machines. User needs to construct a cluster list file that the Prajna application will use.
It is a plain-text file with an ".lst" extension. An example looks like
MyCluster,1005
Machine0
Machine1
Machine2
The first line specifies "ClusterName,Port,Password". ClusterName is a unique identifier of the cluster that the runtime will use. Port is what specified by "-job" argument during client launch. If the client is launched with a "-pwd" argument, please also specify the password, thus the first line becomes
MyCluster,1005,somepasswd
Because currently Prajna uses ClusterName as an identifier and caches cluster's information, when the nodes (or its port) in the cluster changes, please make sure to use a different ClusterName. Otherwise, some strange behavior may happen.
Prajna may use a cluster each different port used by each node. In such a case, the cluster file became:
MyCluster,1005
Machine0,port0
Machine1,port1
Machine2,port2
Prajna may also use a cluster with multiple daemon run on each node. This is typically used for situation when there are multiple physical resources on one node. For example, in a GPU cluster, a physical machine may have multiple GPUs, a Prajna daemon can be started for each GPU.
Use a tool called PrajnaClusterStatus.exe
- For user of the Nuget package, it is located in subfolder "tools\ClusterStatus"
- For user of the GitHub source code, after the source tree is built, it is located in subfolder "bin\ReleaseX64\ClusterStatus" or "bin\DebugX64\ClusterStatus"
If the tool is able to show the disk usages etc on the target machines, it means the deployment is successful.
Prajna applications can construct the cluster object by supplying the path to this list file (with C#):
var cluster = new Cluster(pathToLstFile);This section describes how to use provided Powershell scripts to deploy and launch the Prajna client on a cluster of windows domain-joined machines.
- For user of the Nuget package, the scripts are located in "tools\Scripts\WindowsDomainCluster"
- For user of the GitHub source code, the scripts are located at "scripts\WindowsDomainCluster"
For all target machines
- User needs to have a credential with admin privilege
- PowerShell remote execution is enabled
- The ports that will be used by the client are opened
The scripts like "Deploy-Clients.ps1" assumes user has the same credential for all target machines, and uses the same ports and the same deployment location for all machines.
In PowerShell, cd to the folder where the scripts locate, and invoke "Deploy-Clients.ps1"
PS> .\Deploy-Clients.ps1 -ComputerNames @("Machine1", "Machine2", "Machine3") -SourceLocation ..\..\Client -ClientLocation C:\Deployment\PrajnaClient -Port 1005 -JobPortRange 1250-1299
- -ComputerNames specifies an array of target machines
- -SourceLocation specifies the absolute or relative path to the folder that contains the client
- -ClientLocation specifies the path to put the client on target machines
- -Port specifies the port for the client to accept job requests
- -JobPortRange specifies the range of ports to be used by the client
To enable authentication, add an extra "-Password" parameter to pass in a string of desired password.
If an explict credential is required for executing powershell command on remote machines,
PS> $Cred = Get-Credential
It will prompt a dialog for input the credential. Here also assumes that the credential is the same for all remote machines. Then
PS> .\Deploy-Clients.ps1 -ComputerNames @("Machine1", "Machine2", "Machine3") -SourceLocation ..\..\Client -ClientLocation C:\Deployment\PrajnaClient -Port 1005 -JobPortRange 1250-1299 -Cred $Cred
Once the deployment is done, user can construct a Cluster List File, verify the deployment, and construct a cluster object in code by referencing the cluster list file.
- Start-Clients.ps1 : start clients on machines by specify the location of the clients on the machines, for example
PS> .\Start-Clients.ps1 -ComputerNames @("Machine1", "Machine2", "Machine3") -ClientLocation C:\Deployment\PrajnaClient -Port 1005 -JobPortRange 1250-1299 -ShutdownRunningClients $true
- Stop-Clients.ps1: stop clients on machines by specify the location of the clients on the machines
PS> .\Stop-Clients.ps1 -ComputerNames @("Machine1", "Machine2", "Machine3") -ClientLocation C:\Deployment\PrajnaClient
- Start-Client.ps1 : start the client on a single machine
- Stop-Client.ps1: stop the client on a single machine