8

Let's considering the following scenario, an investment firm runs a computer cluster(using red hat as os), some computation which involves some company's trading secret will run on it, there is a SA who is responsible for maintaining the cluster, but the firm think it is best to keep the trading secret to just a few very senior people(basically cofounders) here, what is the best way to let the server administrator to maintain the cluster without being able to touch the trading secret?

Update:

In this case the trading secret means source code and programs.

these programs are building blocks for profitable trading algorithms that competitors in the market would like to pay to gain insights(and this does happen), so even if the admin don't know/don't understand the source codes or don't know how to do with the programs, someone else, with professional skills will happily to do that part of job.

The cluster has 30-40 computing nodes, is it feasible to let a cofounder to be a system admin without taking all his time on it?

Some of the cofounders have sufficient cs background to write efficient code for supercomputers, but none has system admin experience.

  • 1
    If there is a sysadmin/someone with technical knowledge in these "few very senior people", you could set up a separate server cluster and only give that person administrative access to it. Otherwise, it's impossible - you can't hide things from root. – André Borie Sep 06 '15 at 16:33
  • 2
    Why would you want to - that's the important question. –  Sep 06 '15 at 19:29
  • 1
    What do you mean by "Trading secret"? is that a secret process, or a secret key? please explain what you mean. It might be possible not to hide such stuff but to encrypt it. – hookenz Sep 06 '15 at 21:40
  • 2
    Some of the answers are very informative on topics of server administration, but the aspect I see some of them miss is that OP seems to be aware that there is a threat of the admins being compromised. Perhaps it has happened elsewhere, perhaps it has happened to that company. So there needs to be a defense within this new cluster from that threat. Or perhaps in my ignorance of SELinux, ACL's etc I don't see how all (seven as of now) answers address this threat. Methods that only detect a problem after it happens seem to miss what I would be looking for were it me. –  Sep 07 '15 at 02:58
  • 1
    Correct. This is probably the reason this post was migrated here: "how do I make sure my sysadmin can't steal my secrets?" is a question with no simple, sure and foolproof solution (other than "hire comeone you can trust"). – Massimo Sep 07 '15 at 12:10
  • 1
    Just make sure the server admin's workload is high enough that they don't have time to poke around in your secret stuff? – Dan Henderson Sep 07 '15 at 14:44

7 Answers7

19

No, as a rule of thumb you can't completely block administrators and still allow them sufficient access to do their jobs. You can throw up some barriers that make it difficult for an individual abuse their privileged access (and remain undetected) without conspiring with others.

Most companies deal with these kind of challenges in more or less the same way: they mitigate the risks with procedures and controls, enforced by contracts, technology and auditing.

  • you hire capable and trustworthy people and offer them sufficient incentive to continue to act in that fashion (the carrot: interesting work, appreciation, cash and the stick: penalty clauses, prosecution, etc)
  • after initial setup administrative access is restricted (think very limited sudo instead of a known root password)
  • use tooling so your administrators don't need actual access to the systems commandline to perform common administrator duties
  • privileged account management, every privilege escalation follows procedure and requires a strict four eyes principle
  • No single individual gets to hold all the keys to the kingdom
  • trust but verify, implement independent auditing
  • separation of duties, a sysadmin does not have DBA rights, nor vice versa
  • encryption of data at rest (makes it difficult to get to your critical data by by-passing internal access controls in your application by reading it directly from disk)
  • SELinux mandatory access controls can really restrict "the power of root"
  • and I could even imagine a setup where every administrator action is scripted, rather then manually executed.

So there is no single measure, but a combination of the measures I mentioned and many I may have skipped, can make it difficult for a sysadmin to get access to data he/she should not, at least not without discovery.

HBruijn
  • 1,345
  • 10
  • 13
  • 6
    I think the key is to recognise you can't stop someone but that you can put something in place to detect that it has been done. –  Sep 06 '15 at 21:46
7

If an user has full administrative access (root on Linux, Administrator on Windows), he (or she) can do whatever he wants, up to and including modifying file permissions and access rights if for some reason there is something he doesn't already have access to. This is unavoidable, because it's the whole point of having full administrative access in the first place.

The solution is to not give the user full administrative access, but only to grant him the required permissions for managing what he needs to manage; however, depending on the systems and the tasks he actually needs to perform, this can get tricky and can range in difficulty from something as simple as "place the user in a specific group" to "heavily customize file system permissions and system policies".

Massimo
  • 721
  • 5
  • 13
5

If the application's data and code is to be kept confidential, then I suggest handling data in an encrypted form wherever possible.

Granted, an admin could take measures against that, but it's still a better approach than just relying on system permissions only.

Auditing an admin's actions with auditd by another team might be more effective than crippling an admin account and making day-to-day tasks difficult.

fuero
  • 531
  • 4
  • 7
3

You might be able to hide something from a system admin if you are willing to spend some money doing so (this may not be much of an issue since your scenario refers to investment firm trade secrets) and accept some other risks.

The basic idea here relies on implementing a BIOS resident rootkit that hides access to a specific functionality in the OS. Since you probably have modern servers, they would be equipped with UEFI, a newer form of BIOS. The UEFI firmware supports loading a piece of code referred to as an Update Capsule.

On Windows-based machines this functionality has been used by companies like Lenovo to install persistent software, as can be read here and here. An article by Intel that describes this functionality on Linux based machines can be found here.

The Update Capsule would contain the "rootkit", in this case a custom piece of code that would always load when the servers are started and it would have some components that help monitor each other (to ensure that the UEFI part has not been overwritten, etc).

Exactly what access to the OS this code would have to block depends on the nature of the trade secrets (whether they are some running code, values in some DB, or something else).

Whether this would be the right solution depends on factors such as the exact nature of your environment (whether you use VMs or not, what kind of fail over configuration you have, etc).

And then this also comes with the caveat that nothing is absolutely foolproof. So you would for example have to trust the developers (or hire some other people to check the source code for e.g. back doors). Also if the system admin has physical access to the hardware (the actual servers) where the trade secrets reside, he may be able to get around this setup.

1

There're few things you can do

  • restrictively grant permissions to sysadmin
  • encrypt confidential data
  • NDA agreement with sysadmin
  • store data on a remote system where only trusted people have access
Rick Chatham
  • 234
  • 1
  • 13
Mateusz Pacek
  • 11
  • 1
1

Are the development computer and the production computer the same? If not, you can keep the source code on a (highly restricted access) computer, compile using code obfuscation, and use the compiled code on the production cluster, with the code needing a PW before running.

Shane Di Dona
  • 111
  • 2
  • 1
    you could actually go a step further than that, and write the source code in a language that the sysadmin does not know AND is hard to read even if you know other programming languages. Something like Golfscript, Pyth, Befunge, APL, Brainfuck if you're feeling adventurous, Malbolge if you're feeling sadistic... – Nzall Sep 06 '15 at 21:12
  • 1
    Seems like a lot of work... – Shane Di Dona Sep 07 '15 at 01:11
  • @ShaneDiDona My reading is that budget is not the issue (which translates to amount of work is not the issue) because the nefarious competitor will spend more and failure is also costly. –  Sep 07 '15 at 03:03
1

If you want to have someone other than yourself manage the systems that contain sensitive information. It boils down to hiring the right Administrator that you can trust and has the ethics to not abuse his or her powers to protect company data. Also it requires having the agreements about not disclosing such information and having legal repercussions should there be an ethical lapse.

Having been a Jr. Systems Administrator at a software development company my job was to maintain the critical systems. Many of these systems contained the source code, sensitive e-mails and financial data of the company. Having that power, I made it my job to protect the data from those that where not given the permission to view said data. Also being a Systems Administrator, I would not look at the data because it was out of the scope of my work. If having to look at some of this data was necessary to perform my job, I would ask for authorization before doing so.