Their names - in the history of the Institute

V.V. Mitsyn: "At LСTA we were pioneers in the grid..."

In mid-December last year, we recorded the memoirs of Valery Valentinovich Mitsyn, a senior researcher at the laboratory. To the great regret and grief of his colleagues, Valery Valentinovich did not live to see the day these lines were published. MLIT JINR and the entire world community of computer scientists working in the field of grid technologies have lost a high-qualified unique specialist in various fields of advanced information and network technologies. And today, we are publishing Valery Valentinovich's memoirs about his life path and colleagues.

We moved to Dubna from the village of Severny of Taldom District in 1964, when I was 12 years old. The city was compact and green, where there are new buildings today, where we watched beavers, went for mushrooms and berries, built huts. The city has been quite green since then, I live on Mokhovaya Street and the forest is very close.

In 1975, I graduated from the Faculty of Computational Mathematics and Cybernetics of Moscow State University and decided to go further - to the South Urals. The places there were beautiful - mountains, lakes but unfortunately, there was practically no work, since the supply of new BESM-6 was just waiting.

We studied documentation, went on business trips to different institutes. I also visited JINR, G.L.Mazny and his colleagues to understand what BESM-6 was for system programming. After having worked for two years in the Urals, I returned to Dubna: at first, I entered the newly opened Research Institute "Atoll" but the computers were not the same as at the Joint Institute and afterwards, I moved to the Department of Radioelectronics, located on the DLNP site but not included in the JINR structure. There were more serious machines - SM-4, an analogue of the American PDP-11, which I have already managed to work on as a programmer, system programmer, administrator. Thanks to Evgeny Mazepa, we have a remote JINR terminal - it was a great progress and many thanks to Evgeny for that. Via this terminal, I became friends with LCTA employees - engineers and networkers that "dragged" me to the laboratory. My classmate Sergey Kadantsev worked at LCTA. There was a more serious technique and more strenuous work. At first, we had EU computers and later, VAX machines appeared, they were closer to me because I had already been involved in the work on them. Together with Volodya Subbotin, we served the operating system (OS), later, Mikhail Popov and Anatoly Lomov joined us. A team gathered that was interested in more advanced operating systems. The VAX was powered by the VAX VMS system that was not used anywhere else and however, was also on IBM and EU computers.

Meanwhile, operating systems appeared that could work on different types of machines. This is a UNIX-type OS, not quite freely distributed but it was possible to install and work on it. A little later, the CONVEX machine appeared. At that time, it was a monster: it had a regular processor, like advanced machines and eight processors that performed parallel tasks. It, by the way, interested physicists, they used this parallelism. The processors were so secret from the point of view of the Americans that every three months, an employee came from the Embassy and checked if they were all there. In the late 1980s and early 1990s, some kind of rapprochement began with our American, as they were then called, partners and Digital machines began to appear in our country. We also had to constantly contact engineers because the machines often broke down, sometimes, at night and we went to work with them to restore systems. CONVEX became practically the main machine for users - first, in LCTA, later, in other JINR laboratories. UNIX machines began to gain popularity among both system administrators and users. On its basis, a connection with the Internet later appeared: at first, as it were, not entirely legal, through the closed telephone system "Iskra" and with our IP addresses we could not go outside the country. And when Evgeny Mazepa took over the network service, IP addresses appeared that are still used in JINR. And we formed a team, although Volodya Subbotin had already left the Institute for a commercial organization and together with Mikhail Popov and Anatoly Lomov, they began to be engaged in UNIX OS on PC-type machines. I got, in my opinion, 20 floppy disks with the first installation version of UNIX.

JINR introduced personal computers. Of course, they played different games on them but you could do serious things. Then there was a lull in LCTA and I went on a six-month business trip to Germany, worked at DESY. I was mainly engaged in helping users to transfer programmes from an OS version to another. And Anatoly Lomov also worked there but he was a Windows specialist. DESY began organizing clusters of UNIX-based personal machines, that is, connecting them to operate together. A dozen machines could provide about the same power as a VAX machine. For some reason, this undertaking did not receive distribution in DESY.

Six months later, I returned to Dubna and in the early 2000s, not a grid began, at first, it was the project "Monarch", to unite computer centres into a single distributed system for launching tasks and storing data. At first, the ideas were completely crazy: since the IP networks between these centres were not fast, they wanted to transport data from one centre to another on cassettes. But quickly enough, networks began to meet the requirements of distributed data processing and storage and computing centres also became close to this idea of participation in distributed computing. And then Vladimir Vasilievich Korenkov that headed our department, began to give out ideas one crazier than the other. Then we had a communication channel to Moscow with a bandwidth of, it seems, 10 Mbps, later, it became 50 Mbps and he offered to participate in the grid, in the LCG project (a set of programmes that provide work on tens of thousands of machines of collaboration experiments at LHC - O.T.). We cheered up, tried to do something and we immediately succeeded. Although the idea seemed crazy at first. We at LCTA JINR were pioneers in the grid along with CERN and in some other large centres and the first - in Russia. At first, the United States did not really participate in it, there were theorists. Afterwards, the Kurchatov Institute, the Research Institute of Physics and Technology of Moscow State University joined us. The idea turned out to be not only tempting, but also promising, although at first, there was not very much technology and the network was not very fast. Gradually, resources increased. On our second floor, there were measly 15 "personnel" united in a local network and included in the grid. In addition, they were used by mathematicians and physicists at JINR, that is, the system was complex. Then a tape robot appeared but it operated extremely poorly. It was still possible to write information into it but it is almost impossible to count from it.

It was the idea of V.V.Korenkov and he supported and developed it in every possible way. As a result, LCTA Directorate, then MLIT, was imbued with it and the central directorate began to support it. We got some resources: one rack with Hewlett-Packard servers, then several. We found a company that supplied us with American equipment. Today, unfortunately, it is directly impossible but something comes to us and what is interesting: some equipment supplied 12-15 years ago still works, bought 10 years ago - half out of order and what we received five years ago is good if we repair it under warranty.

Adjusting the first clusters. I.M.Tkachev behind the terminal.
N.S.Astakhov, T.A.Strizh, V.V.Mitsyn, V.V.Korenkov are standing. 2007.

Gradually, we developed as part of a grid - a Tier-2-level site. Basically, we had calculations, the amount of data storage was small. But another crazy collective idea arose together with the Kurchatov Institute Research Centre - to develop a Tier-1 level website. Then there were eight of them in the world, currently, there are fourteen. This idea was approved and we conceived to make the site distributed. We provided support for the LHC CMS experiment and KI - for the other three experiments. The work was funded from the federal target programme, in which KI and we were equal partners, since it turned out that we have both counting resources and storage resources more than they. When V.V. Korenkov convinced the central directorate that this project would give JINR fame, prestige and experience, it means a lot for future experiments of the Institute, then funds in MLIT began to flow in greater volume and equipment purchases increased. We did not even always have time to install the equipment on time, since the efforts of engineers and system administrators were always limited. Quickly enough, we overtook KI in terms of resources and turned out to be one of the best Tier-1 centres in the CMS experiment. There were good relations with Moscow and a good network with KI, afterwards, they "broke through" the communication channel to Amsterdam and later, to CERN. So, we have been working since then.

At first, we had one division in MLIT for computing and storage resources - our team itself. Later, a cloud infrastructure developed that we could no longer cover but young people appeared that took it into their own hands. Afterwards, the hybrid computing appeared, they were also taken up by the young. HybriLIT currently has quite large resources - material and human. Both teams work extensively and well. JINR began to participate not only in the LHC experiments, but also in Chinese and American ones. The data was stored on our infrastructure. The NICA project developed, at first, at the level of simulation but requiring large counting resources - ours, Tier-1, Tier-2, cloud infrastructure and the Govorun supercomputer.

In the MICC monitoring and control room. 2021

Presumably, the current resources, especially those that we still increase, will satisfy the needs of old experiments and new ones. There was a problem with the installation of new resources - the hall of the second floor was not dimensionless. The equipment of the cooling and ventilation systems could no longer provide new computing and data storage capacities. Almost on the go, repairs were carried out, it was a feat of engineering services - many components of the infrastructure were updated. Today, it came to the hall on the fourth floor, where the EU computers once stood. There is a project, work begins on equipping the hall and we hope that next year, a new place will appear for the installation of new resources.

There were two main projects - LCG and WLCG, one passed into the other, they provided both computing and data storage, the rest of the grid projects were semi-administrative or administrative. And the RDIG project was developed in order to finance the development of the grid not only from JINR, but also from Russia and indeed, it turned out to purchase a batch of unified equipment that was supplied to Russian centres participating in WLCG. Thanks to which, by the way, the Higgs Boson turned out to be a reality and did not remain a theory.

Oddly enough, over the past 15-20 years, practically nothing has changed - the same Fortran, the same software products. It is difficult to move physicists but 5-8 years ago, young people appeared, Indian programmers professing other approaches using different software. Users write in the simplest languages but it requires more RAM, faster machines are required but programmes are still on Fortran. Our people that worked at Fortran on BESM-6 40 years ago may well programme something at WLCG. The UNIX-LINUX OS won, it is installed on smartphones, it operates on supercomputers. Today, it is the basic OS and Windows loses its position.

Something changes - authorization for secure access to remote resources, versions of operating systems change and you have to switch to a new version every few years, since the old one is no longer supported. Hackers are not asleep either, trying to find new mistakes. True, recently I have seen that they try to hack new versions, they do not reach the old hands, or what? In distributed computing, in recent years, devices, such as Govorun, have begun to be connected, although it is not always easy to do. I doubt that there is a big gain from it since distributed computing is a different organized system. In short, evolution is underway.

If we compare the working atmosphere, then before, the staffs were small and friendly. Today, the group that deals with computing and storage resources is no longer a single staff: someone is engaged in a grid, the others - in a cloud service, the third - in Govorun. We began to communicate less with each other, less common holidays but by and large, the atmosphere in our staff has always been friendly and the administration treats us well. Previously, meetings were regularly held at which "steam was released", sometimes, pokes and slaps were received from the authorities and it also rallied. Maybe email has replaced them today. Nevertheless, we always try to meet problems together, I share my experience with young people when I see that they have some difficulties.

There have been quite a few amazing moments in our work: when it seems that it is far from meeting the problem and it will be met with difficulty, suddenly, there is a way out quickly. It is probably expected, especially over the years, as experience has been gained, especially, working on different platforms. Not losing it is also a problem. I say to young people, "Guys, learn, you will know a lot." They do not need to be afraid of seemingly unsolvable problems, do not lock themselves in a narrow area, try to "dig" around. As a rule, if not 90 percent, then 70 percent of this knowledge will someday come in handy. The system administrator needs to have a large amount of knowledge, to be able to meet small and large problems, sometimes, non-standard solutions are required.

I wish the young people not to stop in their development, not to dwell on one task, even if you can defend a thesis on it, to look around, to grab pieces of what can then come in handy and most likely, they come in handy. Another principle to remember is that the main thing is not the goal itself, but the path to it. And you don't ever have to give up, what doesn't work out will definitely work out. I hope that most of my colleagues also work according to these principles, consistently achieving their goals. I wish everyone only the good in New Year, so that everyone is happy!

Recorded by Olga TARANTINA,
photos from MLIT archive