Week 11 — Mobility of data and computation

This week's topic is about mobility: moving your data, and your computation, around the Internet.

Data mobility

• file transfer with sftp/scp
• file sharing
• cloud concepts

Computation mobility

• The network is the computer.
• Remote login with secure shell.
• Client/server architecture for graphical displays; X11; remote desktop protocol; VNC.

Deliverable

• download a text file from a server, modify and edit it
• using command-line tools, upload the result

Evaluation

Up to 10 points can be gained towards your final score by completing the in-class exercises on Friday.

What you will learn from this class

• What the Internet is.
• What a Network Interface Card is and how NICs are identified by their MAC addresses.
• How wires, cables, and WiFi connect devices together to form a Local Area Network.
• How devices on the Internet are identified by their IP address.
• How the Domain Name System translates names into numeric IP addresses.
• How data is broken into packets and routed through the Internet from sender to destination.
• The difference between the Transmission Control Protocol and the User Datagram Protocol.
• How reliability and performance tradeoffs can be made by choosing TCP or UDP.
• How to obtain information about your computer's NIC, IP address, DNS server, etc.
• How to use ping, traceroute, ifconfig, and nslookup to diagnose network problems.

Preparation

This week's preparation is to watch some short videos about Internet services that help with mobility of data and computation.

Notes

FTP: file transfer protocol

What is FTP?      https://m.youtube.com/watch?v=wig1szO7en8

Summary:

• File Transfer Protocol (FTP) uses TCP as its underlying transport protocol.
• FTP allows files or data to be transferred between any two computers on the Internet.
• Because it uses TCP, all transfers are reliable.
• FTP uses a “client-server” model: the remote computer (server) stores the files, and your computer (client) transfers files to or from the server.
• When connecting to the server, the client user (you) must first authenticate (log in).
• If the connection is accepted, the client (you) can send commands to list directories, upload (send) or download (receive) files, and so on.
• You can interact with FTP using a graphical client, which looks a bit like the Windows Explorer or Mac Finder.
• You can also interact with FTP using a command-line interface typing actual FTP commands yourself, which is much more flexible.
• FTP is useful for large file transfers or bulk uploads that are impossible with e-mail or online file sharing services.
• FTP is fast and uses very few resources on the server.
• FTP is often used by web site hosting services (Wix, GoDaddy, Weebly, etc.) for users to upload their web site content.

Useful FTP client commands

Exercise: practice using FTP on a public test server

You can practice FTP using a public server providied by speedtest.tele2.net. The example shown here uses the Windows FTP client, but it will work the same on Mac or Linux. The parts typed by the user are shown with a red background. (Click on the image to see it at the original size.)

1. Connect to the server using this command: ftp: speedtest.tele2.net
2. When asked, enter the user name: anonymous
3. When asked for a password, press Enter (to leave the password blank).
4. Use ls or dir to obtain a listing of the public FTP directory.
5. Make sure you are transferring files in binary mode with this command: bin
6. Download a test file using this command: get 10MB.zip
7. Close the session with this command: bye

(Note that these zip files contain junk and there is nothing you can recover from them.)

Exercise: view an FTP server using your web browser

Most web browsers can connect to and view FTP sites.

1. Visiting this URL in a browser tab or window: ftp://speedtest.tele2.net/
2. Try clicking on the upload directory to enter it.
3. Leave the window open while you complete the next exercise.

Exercise: practice uploading a file to an FTP server

1. Create a small text file called test.txt containing a few words of text.
2. Connect to speedtest.tele2.net using your command-line FTP client, as shown above.
3. Once connected, use the cd upload command to change to that directory.
4. Use the bin command to change to binary mode.
5. Use the put test.txt command to send your text file to the server.
6. Refresh your browser window; you should see your file on their server.
7. Click on test.txt in your browser, download the file, and verify it has your few words of text inside it.

Exercise: practice using FTP on your own Windows machine

SSH: secure shell (remote login) and secure copy

Using SSH and SCP:      https://m.youtube.com/watch?v=rm6pewTcSro

ssh is stands for secure shell. ssh is a way for you to remotely access a computer with full command line access.

To connect to a remote machine, use ssh username@address where

• username is your account on the remote machine and
• address is the IP address or DNS name of the remote machine.

The default port of ssh is 22, but for security reasons many administrators choose to run ssh on a different port. To change port that you are connecting to (e.g., port 1234) use the -p (port) option:

ssh -p 1234 username@address

Once connected, ssh will prompt you for your password on the remote machine. Type the password to authenticate and log in. If successful you will have full command-line access to the remote machine, just like running a terminal on that machine.

One thing you can do remotely is to find the full path name for a file that you want to copy securely to your local machine using scp.

scp stands for secure copy. scp is a way to move files between computers that is more secure than FTP. It can copy individual files, multiple files using wildcards, or an entire directory (recursively).

The syntax of scp is just like the cp command:

scp fromPath toPath

If you are copying a directory recursively, add the -r (resursive) option:

scp -r fromDirectoryPath toPath

In either case the toPath can be a filename (which is replaced if it exists) or a destination directory name.

Either fromPath or toPath (but usually not both) can be a remote file. The syntax for a remote path is username@address:path where

• username is your account name on the remote machine,
• address is the IP address or DNS name of the remote machine, and
• path is the path to the remote file, either absolute or relative to your home directory on the remote machine.

(Note that if you want to guarantee that path is the name of a directory then put a trailing / after it as shown in the examples below.)

If the ssh port on the remote machine is not the default 22 then add the -P (port) option (note the capital letter).

Assuming my remote username is piumarta, the remote machine is called server, and ssh is running on port 1234, then

scp -P 1234 piumarta@server:/tmp/data.txt /tmp/

copies /tmp/data.txt from the remote machine server into the directory /tmp on the local machine

scp -P 1234 /tmp/data.txt piumarta@server:data/

copies /tmp/data.txt from the local machine to the data directory located in my home directory on the remote machine server.

scp -P 1234 -r $HOME piumarta@server:/var/backups/laptop

copies my entire home directory recursively to /var/backups/laptop on the remote machine server.

The last example makes a backup of all your personal files. However, it is not an efficient way to back them up because it will copy every file — even those that have not changed since the last backup.

Synchronise files/directories: rsync

The rsync command synchronises two files or directories. The files or directories can be local or remote. The syntax is the same as scp and the -r option (recursive) is used to copy directories.

One popular use of rsync is to make backups of important files. Compared to scp, the advantage of rsync is that only changed files are copied to the destination. The first time you make a backup using rsync can take a long time, but subsequent backups are much faster because they copy only the changes you made to your files. Even very large files are copied efficiently because only the changed parts are updated.

How to make backups using rsync:      https://m.youtube.com/watch?v=8d5B1JC-1d4

Remote desktop

A remote desktop gives access to the graphical user interface of a remote computer over the Internet. A user can interact with the remote system as if they were sitting in front of it. Keyboard and mouse inputs are transferred from the user's computer to the remote computer, and screen updates are transferred from the remote computer to the user's computer.

Typical uses include using a workplace workstation when at home or vice versa, fixing a computer problem remotely, performing administrative tasks easily, and demonstrating processes or software applications. In addition, headless computers that have no monitor, keyboard, or mouse can easily be accessed remotely by administrators.

Protocols for remote desktop connectivity include:

Remote Desktop Protocol

RDP is built-in to Windows professional and higher versions. One disadvantage is that only one person can access the controlled computer at once. When a remote user connects to it, the local user is locked out.

Virtual Network Computing    https://tightvnc.com/

VNC is available on (and interoperates between) Windows, MacOS, and Linux. One advantage is that multiple users can connect to the same remote screen, and a user sitting at the remote computer is not locked out when remote users connect.

How to install VNC:      https://turbofuture.com/computers/How-to-Install-and-Configure-TightVNC

How to use TightVNC:      https://www.youtube.com/watch?v=x9xTyh63Tos

ネットワーク経由でWindows 10を操作する（VNC編）:      https://www.youtube.com/watch?v=98rQ9J5XE_g