YAMI C++ Wrappers Tutorial

Copyright © 2001-2003 Maciej Sobczak



Contents

Introduction

If you already know the YAMI Core Library and you have read its tutorial, you will quickly notice that this text is almost exactly the same. The only substantial change is the source code. This apparent author's laziness is to show you that the YAMI C++ Wrappers are in fact very thin wrappers around the core library. Do not hesitate to run the examples from both tutorials in pairs to see that YAMI provides abstracts that are language-independent and that the connectivity between programs written in different languages is really achieved.


In this tutorial, there are two assumptions:

Echo

As a first example, you will write your own echo server and client. The echo server will wait for messages from clients and will print them on the screen. Apart from that, the server will also accept a special shutdown message. There will be no data sent back to clients.

Easy client

First, you will learn how to write simple client. In most distributed systems it is easier to write clients than servers.
The first client in this tutorial sends a message to the server that tells it to shut down. Here it is (file shutdown.cc):

  1 #include "yami++.h"
  2 #include <iostream>
  3 
  4 using namespace YAMI;
  5 
  6 int main()
  7 {
  8      netInitialize();
  9     
 10      easySend("127.0.0.1", 12340, 2,
 11           "echo", "shutdown");
 12     
 13      netCleanup();
 14     
 15      return 0;
 16 }

Yes, it is that easy.


The first line is necessary to use YAMI in C++.

The line with call to easySend alone is enough to send a message to the server. There are five parameters used in this function:

Note that the easySend function is overloaded and that there is another version of this function that accepts additional parameter. We will use it later.

Note also the netInitialize and netCleanup functions. They are needed on MS Windows system. If you do not plan to write for Windows, you can safely omit them (they are empty for Unix/Linux) but keep in mind that using them will add to portability of your code.

Interesting? So let's pass some data with the message.

Easy client with data

In the previous section, the message had only a name. In many cases it is enough, because it allows the server to make decisions about what action to perform, but there are situations where it would be nice to pass some data together with the message. In YAMI, there are two levels of messages with regard to the data that can be sent:

Both levels are accessible in YAMI C++ Wrappers.

There can be many values of different types that can go with the same message at once. All the values are stored in the so-called Parameter Set, which is basically a list of values of chosen type. Such a Parameter Set has to be created before the message is sent and appended to it. It can be even used many times with different messages, but we will not use this feature here.

The client that causes the server to print ``Hello, YAMI!'' message on the screen can look like here (file hello.cc):

  1 #include "yami++.h"
  2 
  3 using namespace YAMI;
  4 
  5 int main()
  6 {
  7      netInitialize();
  8      
  9      ParamSet params(1);
 10      params.setString(0, "Hello, YAMI!");
 11 
 12      easySend("127.0.0.1", 12340, 2,
 13           "echo", "print", params);
 14 
 15      netCleanup();
 16 
 17      return 0;
 18 }

Yes, it is that easy.


The line 9 defines a local variable that is a Parameter Set object with 1 uninitialized parameter. After that, the parameter is set to be a string of value ``Hello, YAMI!''. As you can see in the next line, the Parameter Set object is used to send the message. Note also that the message name has changed. The Parameter Set object is destroyed automatically.

Full blown client

The examples written so far have one thing in common: they send only one message to the server and quit. There is nothing that prevents you from sending millions of messages using the easySend function, but in terms of performance it will not be a good idea. The easySend function creates a YAMI Agent that is responsible for actually sending a message and destroys it after that. If you want to send many massages, it is a better idea to follow the scheme:

  1. Create the Agent object.
  2. Send the message with the help of the Agent. Repeat this step as many times as you wish.

In this section you will write the client according to this scheme. The client itself will be smarter than before, too - it will read the lines of text from its standard input and send each line as a separate message to the echo server. The code is below (file printall.cc):

  1 #include "yami++.h"
  2 #include <iostream>
  3 #include <string>
  4 
  5 using namespace YAMI;
  6 using namespace std;
  7 
  8 int main()
  9 {
 10      netInitialize();
 11      {
 12           Agent agent(12341);
 13           agent.domainRegister("echoserver",
 14                "127.0.0.1", 12340, 2);
 15      
 16           string line;
 17           while (getline(cin, line))
 18           {
 19                ParamSet params(1);
 20                params.setString(0, line);
 21 
 22                agent.sendOneWay("echoserver",
 23                     "echo", "print", params);
 24           }
 25      }
 26      netCleanup();
 27      
 28      return 0;
 29 }

The Agent is created with its own listening port 12341. This Agent does not use this port, since no information is sent back to the client. Please read about Policies to learn how to create the Agent object without the listening socket.
After the Agent is created, the remote domain is registered in it, like in White Pages Book. The address and port of destination Agent (the one created by the server) is remembered under the name "echoserver" for later use (the name in the address book is arbitrary).
The program performs a loop where it reads lines of text from standard input. Each line is then sent as a parameter with the message to the server.


Note that the sendOneWay function allows to send the message without any response from the server. The responses will be used in later examples.


Note also the artificial scope created by the curly braces. They are to ensure that the Agent object will be destroyed before the netCleanup function is called (such a scope can be created in any other way valid in C++, for example by a function). As noted above, on Unix system neither the cleanup function nor the scope would be necessary.

Server, polling version

Finally, we will write the server.

Err... what does this polling mean? First, write the following code (file servera.cc):

  1 #include "yami++.h"
  2 #include <iostream>
  3 #include <string>
  4 
  5 using namespace YAMI;
  6 using namespace std;
  7 
  8 int main()
  9 {
 10      netInitialize();
 11      {
 12           Agent agent(12340);
 13           agent.objectRegister("echo",
 14                     Agent::ePolling, NULL);
 15 
 16           cout << "server started" << endl;
 17           while (1)
 18           {
 19                auto_ptr<IncomingMsg> incoming
 20                     (agent.getIncoming("echo", true));
 21 
 22                string msgname(incoming->getMsgName());
 23 
 24                if (msgname == "shutdown")
 25                {
 26                     cout << "received the shutdown message"
 27                          << endl;
 28                     break;
 29                }
 30                else // if (msgname == "print")
 31                {
 32                     auto_ptr<ParamSet> params
 33                          (incoming->getParameters());
 34 
 35                     string line;
 36                     params->getString(0, line);
 37                     cout << line << endl;
 38                }
 39           }
 40      }
 41      netCleanup();
 42      
 43      return 0;
 44 }

After the Agent is created, the polling object is registered in it as a potential target of messages. If any message comes with the object name that is equal to the name of the registered object, the Agent will store the message in a queue, one queue for one object. Later, the server program goes into a loop that... asks the Agent if there are any messages for the echo object and processes them. This is why the server is polling - it has to ask the Agent for every new message. Moreover, in this example, the server will block waiting for new message if the queue is empty - the last parameter to getIncoming method is true.
When the message is finally retrieved, the server checks its name. If it is "shutdown", it breaks the loop and terminates. Otherwise (which means that the message name is "print"; in real system you would also take some steps if the message has unknown or unacceptable name), the server gets the Parameter Set from the message and prints its first parameter on the standard output.

Server, passive version

If there was a polling server, then there must be some alternative. It is called a passive server. Previously, the server had to ask for each incoming message. The advantage was that the server could ask in the most appropriate time if it had some other things to do. However, in some situations it is convenient if it is the Agent who tells the server that there is new message. In this scheme, the server itself is passive - it only waits for invocations from the Agent. It is important to note that the invocations are made from the separate thread that belongs to the Agent object. Let's see the code (file serverb.cc):

  1 #include "yami++.h"
  2 #include <iostream>
  3 #include <string>
  4 
  5 using namespace YAMI;
  6 using namespace std;
  7 
  8 class Server : public PassiveObject
  9 {
 10 public:
 11      Server(Semaphore &s) : sem_(s) {}
 12 
 13      void call(IncomingMsg &incoming)
 14      {
 15           string msgname(incoming.getMsgName());
 16           if (msgname == "shutdown")
 17           {
 18                cout << "received the shutdown message"
 19                     << endl;
 20                sem_.release();
 21           }
 22           else // if (msgname == "print")
 23           {
 24                auto_ptr<ParamSet> params
 25                     (incoming.getParameters());
 26 
 27                string line;
 28                params->getString(0, line);
 29                cout << line << endl;
 30           }
 31 
 32           incoming.eat();
 33      }
 34 
 35 private:
 36      Semaphore &sem_;
 37 };
 38 
 39 int main()
 40 {
 41      netInitialize();
 42      {
 43           Semaphore sem(0);
 44           Server servant(sem);
 45 
 46           Agent agent(12340);
 47           agent.objectRegister("echo",
 48                Agent::ePassiveSingleThreaded,
 49                &servant);
 50 
 51           cout << "server started" << endl;
 52 
 53           sem.acquire();
 54 
 55           // the process quits this block
 56           // only when the semaphore is released
 57      }
 58      netCleanup();
 59      
 60      return 0;
 61 }

The most important difference in the main function is that the object was registered as a ePassiveSingleThreaded instead of ePolling. Also, the code implementing the server's functionality is encapsulated in separate class. The last parameter to the objectRegister function is a pointer to the servant object. Its class implements the call function from the PassiveObject interface that will be called by the Agent when there is some new message for the given object. After registering the object, the server process suspends its execution with simple trick - acquire the semaphore that is initially set to 0. In other words, the server process is going to sleep.

Now, let's go back to the servant. The name was chosen because in other distributed infrastructures (notably CORBA) the servant is a piece of software that is responsible for processing messages. This is exactly the purpose of this object. Note that the incoming object is already provided. What does the call function do? The same as before, with few differences:

Asynchronous vs. synchronous

YAMI sends and processes messages asynchronously by default. This means two things:

The asynchronous sending and receiving is not always good, however. Let's say that you run the command:

$ ./printall

(I assume some Unix environment here)

and later type some lines of text by hand. You are not really fast with typing, so the messages (each message contains one line of text) are actually sent to the server before you get your finger off the keyboard. But try this little experiment:

$ ./printall < printall.cc

which should sent as many messages to the server as there are lines in the printall.cc file. After reaching end-of-file, the process terminates... which can happen before the sender thread sends anything! Or you will see only few of the lines on the server side. There are three solutions to this problem:

Similar issues arise on the server side. The messages received from the network are automatically stored in the queue. This queue can get overflown (you can increase its maximum size, but again - do not tell anybody). Similarly, the professional solution is to change the dispatching mode to synchronous by switching off dispatching threads in the Agent.

Look at the printsync.cc and servsync.cc files to see how to do it. If you compile them, you can try the test:

$ ./servsync

in one console and (probably on different computer in the network, but you should change the address used in the client source code) in the other:

$ ./printsync < somelongfile

Enjoy.

One more trick

In the Unix-like systems, the socket can remain open for some time even after the process using it terminated. Try to run the server, shut it down and immediately type:

$ netstat | grep "12340"

You will see some sockets still open in the TIME_WAIT state. It can take a while until they go away, but during that time you will not be able to restart the server - the system will not allow you to bind to open socket. This can get annoying sometimes, but there's a way around. The servsync.cc file will show you, look at the Agent's Policies.

Error handling

Many different things can go wrong in the distributed system. Some of them will be your fault, some not. Almost every YAMI function can throw an exception. In the example programs presented so far there was no exception management - for simplicity and to avoid cluttering the source code. In real programs, you will for sure want to check for and manage the exceptions. The printsync.cc and servsync.cc files are written with simple, but consistent exception checking.

Calculator

In this second example, you will write the client-server pair where the client asks for something and the server responds to the messages, sending some data back to client. This is probably the most common way of developing distributed systems.

The calculator example consists of:

I'm sure that after this example you will be able to write distributed systems of arbitrary complexity.

Client

Let's start with easy things. The client (file calcclient.cc) looks like here:

  1 #include "yami++.h"
  2 #include <iostream>
  3 
  4 using namespace YAMI;
  5 using namespace std;
  6 
  7 const char *serverhost  = "127.0.0.1";
  8 const int  serverport   = 12340;
  9 const int  clientport   = 12341;
 10 const char *domainname  = "someDomain";
 11 const char *objectname  = "calculator";
 12 
 13 int main()
 14 {
 15      netInitialize();
 16      {
 17           Agent agent(clientport);
 18           agent.domainRegister(domainname,
 19                serverhost, serverport, 2);
 20 
 21           ParamSet paramset(2);
 22 
 23           paramset.setInt(0, 100);
 24           paramset.setInt(1, 20);
 25 
 26           auto_ptr<Message> msg(agent.send(domainname,
 27                objectname, "add", paramset));
 28 
 29           msg->wait();
 30 
 31           Message::eStatus status = msg->getStatus();
 32 
 33           if (status == Message::eReplied)
 34           {
 35                auto_ptr<ParamSet> retpar(msg->getResponse());
 36 
 37                int result = retpar->getInt(0);
 38 
 39                cout << "the result is " << result << endl;
 40           }
 41           else if (status == Message::eRejected)
 42           {
 43                cout << "the last message was rejected"
 44                     << endl;
 45           }
 46           else
 47           {
 48                cout << "no correct reply" << endl;
 49           }
 50      }
 51      netCleanup();
 52 
 53      return 0;
 54 }

The main difference to the previous example is that the send method is used instead of sendOneWay. The Message class encapsulates the message token. It allows you to retrieve some information about the message sent to the remote objects. It also allows you to synchronize the client activity with the server - the wait function allows the client to wait for some change in the message's status.

As you can see, the message is sent with two integer parameters in the Parameter Set (100 and 20). The message is sent asynchronously, so that you can continue your job without waiting for response and ask for it later. In this example there is nothing interesting to do anyway, so we decide to wait (with the call to wait) until something interesting happens to the message token. The process wakes up when (for example) the response arrives. The status of the message is examined to find out if there is a real reply or maybe some network error or something else. If there is a reply, we just retrieve the returning Parameter Set and print its only (first) parameter.

The Calculator example shows also that there are two contexts where Parameter Set can be used:

These two contexts allow to send data in both directions.

Easy? So take also a look at the calcclient2.cc file and play with it - it is an interactive calculator console.

There is one thing to remember, though. The wait function is your friend, but do not trust him. It may happen that the server crashes in the middle of the computations. Then - the client does not receive any notification and the status of the message is always ePending. If you call the wait function, you are in troubles (the other solution, sometimes reasonable, is to periodically ask the message token if the response arrived - the client can decide by himself that something went wrong after, say, 10th try). To help you with this problem, the Agent object provides the waker service. The calcclient2.cc file shows how to use it.

Server

The server is presented below (file calcserver.cc):

  1 #include "yami++.h"
  2 #include <iostream>
  3 
  4 using namespace YAMI;
  5 using namespace std;
  6 
  7 const int  serverport  = 12340;
  8 const char *objectname = "calculator";
  9 
 10 class Server : public PassiveObject
 11 {
 12 public:
 13      void call(IncomingMsg &incoming)
 14      {
 15           string msgname(incoming.getMsgName());
 16 
 17           cout << "I have received the message: "
 18                << msgname << endl;
 19 
 20           if (msgname != "add" &&
 21                msgname != "sub" &&
 22                msgname != "mul" &&
 23                msgname != "div")
 24           {
 25                cout << "unknown name - rejecting" << endl;
 26                incoming.reject();
 27                return;
 28           }
 29 
 30           auto_ptr<ParamSet> params(
 31                incoming.getParameters());
 32 
 33           int val1 = params->getInt(0);
 34           int val2 = params->getInt(1);
 35           
 36           int result;
 37           if (msgname == "add")
 38           {
 39                result = val1 + val2;
 40           }
 41           else if (msgname == "sub")
 42           {
 43                result = val1 - val2;
 44           }
 45           else if (msgname == "mul")
 46           {
 47                result = val1 * val2;
 48           }
 49           else /* msgname is "div" */
 50           {
 51                if (val2 == 0)
 52                {
 53                     cout << "dividing by 0 not allowed"
 54                          << endl << "rejecting" << endl;
 55                     incoming.reject();
 56                     return;
 57                }
 58                else
 59                     result = val1 / val2;
 60           }
 61 
 62           ParamSet returnparams(1);
 63           returnparams.setInt(0, result);
 64 
 65           incoming.reply(returnparams);
 66      }
 67 };
 68 
 69 int main()
 70 {
 71      cout << "starting the server" << endl;
 72      netInitialize();
 73      {
 74           Server servant;
 75           Agent agent(serverport);
 76           agent.objectRegister(objectname,
 77                Agent::ePassiveSingleThreaded,
 78                &servant);
 79 
 80           cout << "going to sleep..." << endl;
 81           sleep(0);
 82      }
 83      netCleanup();
 84      
 85      return 0;
 86 }

This is a passive server, because it uses the call-back pattern for message dispatching. You have seen it already in the Echo example.

This server has one design quirk. I show you this, because it is sometimes used in servers working in distributed systems (I have seen it in some CORBA examples, really). Namely - after setting up everything in the main function, the main server thread goes to sleep... and never wakes up. The sleep function with 0 as a parameter sleeps forever. This means two things:

I do not advocate this style of writing servers, although, as I've pointed out, I have seen it. The two clean solutions that I recommend are:

The file calcserver2.cc is the same as presented above calcserver.cc but with simple exception-handling added.

Explicit event processing

This part of the tutorial is aimed at advanced users, who may wish to experiment a little or who find themselves in unusual situations like the necessity to write software on platforms that do not have threads (like classic Unix or some embedded systems). Basically, you will rarely (if ever) need to write your own event processing.

It is possible to compile the YAMI library without threading. Then, most of the library's functionality is gone and it may seem at first that it is not possible to use YAMI at all. But it is.

In order to correctly process connections coming from remote Agents, the local Agent performs a loop, where new connections are managed and where everything is made. In fact, every thread in the Agent performs its own loop, but the only really necessary is the loop in the receiver module. There are two situations where there may be no separate receiver module:

In both situations the programmer needs to perform the loop by himself, usually in the main part of his program - this applies both to client and server. It is also possible to combine this explicit event processing with other loops that may appear in the crucial part of your programs, like in windows-based GUI applications.

The example programs clientloop.cc and serverloop.cc are similar to the examples presented so far, but their Agents are prepared to work without separate threads and the important event processing is triggered explicitly in the application code. Previously, the event processing was hidden in the receiver module which was running by itself.

The interesting thing to note is that the explicit event processing results in best performance, since everything related to message processing is done in the context of a single thread and most (or all) of the synchronization overhead is avoided. Nice, isn't it?. Of course, this comes at the cost of increased complexity, as usual.


Last few comments:


Enjoy! And, of course, tell your friends about YAMI.



Maciej Sobczak 2003-05-05