If I handle the incomming connection directly in the Adruino sketch using YunServer and YunClient I don't need a programm for the Linux Side of the Arduino. Did I understand that correctly? Can I imagine the Linux Side as a special "tool" which passes on the entered values of the smartphone to the Arduino Sketch and which does nothing else?
Yes, the Linux side will actually accept the connections, and pass everything through to the sketch. The necessary code is already in place and set up for you. It is indeed possible to do everything in the sketch and not have to write any code on the Linux side.
There are two ways to do it: as mart256 mentions, a REST API is rather straightforward to set up. An example of that is the Bridge Example shows how this can be done. Of course it can be customized to your needs, and can do a lot more than simply turning pins on and off. With this example, the YunServer/YunClient are working in concert with the web server that is built into the Linux side. When the server sees a web access with a URL that starts with /arduino/ right after the Yun's IP address, it will open up a connection and pass everything that appears after /arduino/ to the YunClient. To send such a command to the Yun, it will be necessary for your Android app to make discrete web requests to the Yun. You can do this manually using a web browser, or your application will have to call the right code to make the request. On many computers, the function curl() is available to do that, but I don't know what's available on Android.
But there is another way to use YunClient/YunServer, and that is as a simple TCP socket server. In this mode, the YunServer listens for connections on a specific port, and when a computer connects to it it creates a YunClient, very similarly to the way the Bridge Example works. The difference is that this is not a web request, but a TCP stream (similar to Telnet.) In this case, your Android application would open a TCP socket to the Yun, and then it would have a persistent socket connection that does not go away at the end of the request - it would stay open. You could send messages back and forth indefinitely, just as if it were a serial port. To use this mode, you change the YunServer call to include the desired port number that it should serve, and you change setup so that it calls server.noListenOnLocalHost() instead of server.listenOnLocalHost() -- not the addition of "no" to the function name.
Now, while you can do it all in the sketch, and ignore Linux, you will be missing out on a lot of the power of the Yun if you do it that way. The Linux side brings so much more to the table, and gives so many more abilities, that you really should stop and take some time (when you have a chance) to learn what it can do. It really does open up a lot more power and capabilities.