I want the client not to change his program,
You cannot completely prevent this when the software runs on hardware that you do not have. To run the software, the CPU must see all the instructions of the program, and they will be saved in the computer's memory.
https://softwareengineering.stackexchange.com/questions/46434/how-can-software-be-protected-from-piracy
Code is data. When the code runs, a copy of this data is insecure code. Unprotected code can be copied.
Overflowing the code with anti-piracy checks makes it a little more complicated, but hackers just use the debugger and delete them. Inserting no-ops instead of calls to check_license is pretty simple.
( https://softwareengineering.stackexchange.com/questions/46434 answers may be useful for you)
The owner of the equipment manages the OS and memory, he can reset everything.
or at least make it "tough enough."
You can make it a little harder.
Write code in Go and compile the software into binary code (may be with obfuscation)
IDA decompiles any machine code. Using native machine code is slightly stronger than bytecode (java or .NET or dex)
Make sure that the program can only be initiated with a secret key that can be sent through the secure port. The secret key may change with time.
If a copy of the same secret key (s) is in the code or in the program memory, the user can unload it and simulate your server. If part of your code or part of the data necessary to run the code is stored in encrypted form and decrypted using such a foreign key, the user can either eavesdrop on the key (after it is decoded with SSL, but before it is used for decryption the secret part of the code) or decrypt the decrypted code / data from memory (it is very easy to see the new executable code created in memory, even with the default tools, such as strace on Linux, just find all mmap with PROT_EXEC flags)
Every time I need to start / stop a program, I can send commands with private keys through a secure port.
This is just a variation of the online license / anti-piracy check (“home phone”)
I think this approach can prevent the root user from using the debugger to reverse engineer my code, or
No, he can start the debugger at any time; but you can make it a little harder to use the interactive debugger if the program often interacts with your server (every 5 seconds). But if he binds so often, it’s better to move some of the calculations to your server; this part will be protected.
And it can still use non-interactive debuggers, tracing tools, and memory flushing. He can also run the program in a virtual machine, wait until an online check is performed (using tcpdump and netstat to monitor network traffic), and then take a snapshot of the virtual machine (there are several options that allow live migration of the virtual machine, only a short pause can be recorded by your program, if it has external synchronization), continue to launch the first copy online and take a snapshot for offline debugging (with all the keys and the decrypted code in it).
re-run the program to check the outputs
Until he breaks the bond ...