Missing-Semester-Notes
Lecture Notes: Missing Semester for your CS education
Introduction
Welcome to the course: The Missing Semester of Your CS Education!
When I first learned about this course at the beginning of my freshman year, I had no concept of command lines, development, and other related topics. Additionally, since the course was taught entirely in English, I tried attending a few lectures but eventually gave up, and my absorption rate was also low. During this winter break, after a semester of experience in my freshman year, I installed the WSL subsystem and mastered the most basic Bash knowledge. When I revisited this course, I felt a sudden clarity. Now, the command line has become a tool I frequently use, greatly improving my learning and development efficiency. Therefore, I will organize my Missing Semester notes in this blog, hoping to inspire and help future learners.
Before starting the learning exploration, I strongly recommend you to ensure all these questions and contents below are carefully read, which will help you gain a better learning experience.
Where to find the course?
On YouTube: https://www.youtube.com/@MissingSemester
The course also has its own websites for clear lecture notes!
For domestic users, you can browse Bilibili for reposting of the original course and Chinese subtitle translation.
What’s this course about?
Let’s take a look at how the course instructor evaluates the positioning of this course.
Missing semester Lec1 Beginning- The overall introduction of this course
This class stems from an observation that Anish, Jose, and I have made while attending various classes at MIT. The observation is that, as computer scientists, we all know that computers are great at performing repetitive tasks and automating things. However, we often fail to realize that there are many tools that can make our own development processes better. We can be much more efficient in how we use our computers because we can use the computer as a tool for ourselves, not just for building websites or software. This class is an attempt to address this issue. It is an attempt to show you some of the tools that you can use to great effect in your day-to-day work, in your research, and in your studies. It’s going to be a class where we want to teach you both how to make the most of the tools that you already know and, hopefully, introduce you to some tools that you don’t know yet. We also want to show you how to combine these tools to produce more powerful results than you might think possible with what you know today.
Chinese Translation:
这门课程源于Anish、Jose和我在MIT参加各种课程时的一个观察。我们发现,作为计算机科学家,我们都知道计算机非常擅长执行重复性任务和自动化工作。然而,我们常常没有意识到,有许多工具可以让我们的开发过程变得更好。我们可以更高效地使用计算机,因为我们可以将计算机作为我们自己的工具,而不仅仅是用来构建网站或软件。这门课程旨在解决这个问题。它试图向你展示一些可以在日常工作、研究和学习中高效使用的工具。这门课程的目标是教你如何充分利用你已经知道的工具,同时也希望向你介绍一些你以前不知道的工具,并展示如何将这些工具结合起来,产生比你今天所知道的更强大的结果。
So this lecture is for Tool-using during your CS education. Several Tools can be seen in the subtitle of each courses.
Am I ready to take the course?
However, I am unwilling to announce the fact that: This course is not suitable for Totally-Green-Hand.🙂↕️🙂↕️🙂↕️ Here are two following reasons I have summarized:
- The course progresses very quickly. If you have no prior knowledge of these concepts, there’s a high chance you won’t keep up with the pace! Although you can pause while learning on video platforms, previewing the basic knowledge before taking the course can provide a better learning experience.
- In learning tools, practice is a very important component. This course mainly focuses on lectures and lacks opportunities for hands-on practice.
Thus, I strongly recommend you to do such things before entering the courses:
Make sure you have installed command-line environment before the course begins! (Most important)
- MacOS or Linux is preferred.
- WSL for Windows
While learning the course content, practice hands-on operations simultaneously!
Still updating~
Lec1-2: Shell Tools and Scripting
$
in Bash
In Bash, $
has special meanings:
- Variable Reference: Access variable values (
$var
). - Special Variables:
$0
: Script name.$1
-$9
: Script arguments.$?
: Exit status of the last command.$$
: Current script’s PID.
- Command Substitution: Capture command output (
$(command)
). - Arithmetic Operations: Perform math (
$((expression))
). - Environment Variables: Access (
$HOME
,$PATH
).
Here $1
is the first argument to the script/function. Unlike other scripting languages, bash uses a variety of special variables to refer to arguments, error codes, and other relevant variables. Below is a list of some of them. A more comprehensive list can be found here.
$0
- Name of the script$1
to$9
- Arguments to the script.$1
is the first argument and so on.$@
- All the arguments$#
- Number of arguments$?
- Return code of the previous command- If the previous command execute successfully, the arguments will probably return
0
. - If it fails, it will return none-0 value.
- If the previous command execute successfully, the arguments will probably return
$$
- Process identification number (PID) for the current script!!
- Entire last command, including arguments. A common pattern is to execute a command only for it to fail due to missing permissions; you can quickly re-execute the command with sudo by doingsudo !!
$_
- Last argument from the last command. If you are in an interactive shell, you can also quickly get this value by typingEsc
followed by.
orAlt+.
find
command
find
is a very powerful command-line tool used to search for files and directories in a directory tree. It supports a large number of parameters and options, allowing searches based on file name, type, size, time, permissions, and many other conditions. Below are some commonly used find
parameters and options, categorized by functionality:
1. Basic Search
.
: Start searching from the current directory./path/to/dir
: Start searching from the specified directory.
2. Search by File Name
-name "pattern"
: Match file names (case-sensitive).- Example:
find . -name "*.txt"
finds all.txt
files.
- Example:
-iname "pattern"
: Match file names (case-insensitive).- Example:
find . -iname "readme*"
finds allREADME
,readme
, etc. files.
- Example:
-regex "pattern"
: Match file names using regular expressions.- Example:
find . -regex ".*\.txt$"
finds all files ending with.txt
.
- Example:
3. Search by File Type
-type f
: Find regular files.-type d
: Find directories.-type l
: Find symbolic links.-type s
: Find socket files.-type p
: Find named pipes (FIFO).-type c
: Find character device files.-type b
: Find block device files.
4. Search by File Size
-size +n
: Find files larger thann
.-size -n
: Find files smaller thann
.-size n
: Find files exactlyn
in size.- Units can be:
c
: Bytes (default).k
: Kilobytes.M
: Megabytes.G
: Gigabytes.
- Example:
find . -size +100M
finds files larger than 100MB.
- Units can be:
5. Search by Time
-mtime n
: Find files modifiedn
days ago.-mtime +n
: Files modified more thann
days ago.-mtime -n
: Files modified within the lastn
days.
-atime n
: Find files accessedn
days ago.-ctime n
: Find files whose status (e.g., permissions or ownership) changedn
days ago.-mmin n
: Find files modifiedn
minutes ago.-mmin +n
: Files modified more thann
minutes ago.-mmin -n
: Files modified within the lastn
minutes.
-amin n
: Find files accessedn
minutes ago.-cmin n
: Find files whose status changedn
minutes ago.
6. Search by Permissions
-perm mode
: Find files with permissions exactly matchingmode
.- Example:
find . -perm 644
finds files with permissions644
.
- Example:
-perm -mode
: Find files with permissions includingmode
.- Example:
find . -perm -u=r
finds files readable by the user.
- Example:
-perm /mode
: Find files with any of themode
bits set.
7. Search by User and Group
-user username
: Find files owned by the specified user.-group groupname
: Find files owned by the specified group.-uid uid
: Find files owned by the specified user ID.-gid gid
: Find files owned by the specified group ID.
8. Logical Operations
-and
or-a
: Logical AND (default).-or
or-o
: Logical OR.-not
or!
: Logical NOT.()
: Group conditions.- Example:
find . \( -name "*.txt" -o -name "*.md" \)
finds.txt
or.md
files.
- Example:
9. Execute Actions
-exec command {} \;
: Execute a command on the found files.- Example:
find . -name "*.log" -exec rm {} \;
deletes all.log
files.
- Example:
-exec command {} +
: Pass multiple files to the command at once.- Example:
find . -name "*.txt" -exec cp {} /backup/ +
copies all.txt
files to/backup
.
- Example:
-ok command {} \;
: Similar to-exec
, but prompts for confirmation before executing the command.-delete
: Delete the found files.-print
: Print the path of the found files (default behavior).-ls
: Display the found files inls -dils
format.
10. Other Common Options
-maxdepth n
: Limit the maximum directory depth for the search.- Example:
find . -maxdepth 2 -name "*.txt"
searches for.txt
files only in the current directory and its immediate subdirectories.
- Example:
-mindepth n
: Limit the minimum directory depth for the search.-empty
: Find empty files or directories.-readable
: Find readable files.-writable
: Find writable files.-executable
: Find executable files.
11. Comprehensive Examples
Find and delete all .log
files in the current directory:
1 |
|
Find files larger than 100MB in /var/log
and display detailed information:
1 |
|
Find .txt
files modified within the last 7 days in the current directory and copy them to /backup
:
1 |
|
Find and delete all empty directories in the current directory:
1 |
|
Find files with permissions 644
in the current directory and change their permissions to 755
:
1 |
|
grep
command
grep
is a powerful command-line tool in Unix/Linux used to search for specific patterns within files or input text.
1 |
|
pattern
: The text or regular expression to search for.file
: The file(s) to search within. If omitted,grep
reads from standard input.
For example, you can using the grep
command by using pipes.
1 |
|
This command will first output the file contents of sshd_config
while the content itself won’t be printed directly into the screen. It will be transmitted as the input of the grep command.
Common Options
-i
: Ignore case (case-insensitive search).
1 |
|
-v
: Invert match (show lines that do not match the pattern).
1 |
|
-r
or-R
: Recursively search directories.
1 |
|
-n
: Show line numbers of matching lines.
1 |
|
-c
: Count the number of matching lines.
1 |
|
-l
: List filenames containing the pattern.
1 |
|
-w
: Match whole words only.
1 |
|
-A
,-B
,-C
: Show lines after, before, or around the match.
1 |
|
For example, I have a directory (current) and exists files as below:
1 |
|
Now I can use several commands as below to efficiently search specific information that I want!
I can use -r
to search for the whole current directory.
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
You can make the command line more complex to get more specific data!
1 |
|
1 |
|
More specific:
1 |
|
Using the -h
options to forbid grep showing filenames while searching the directory recursively. Adding more pipes and using Regex can enable developers to make more specific pattern matching problems more freely.
1 |
|
By using this command, you can get all the mottos for all person’s files, which will significantly enhance one’s working efficiency. For more options, you can typing man grep
for advanced usage.
Lec3: Vim editors
You can just go to Vim tutorial for help.
Lec4: Data Wrangling
Lec5: Command Line Environment
Lec6: Git (Version Control)
Lec7: Debugging and Profiling
Debugging is of great importance! Mastering debugging in the coding process will significantly enhance your coding speed and time you will cost when your codes get an error.
“The most effective debugging tool is still careful thought, coupled with judiciously placed print statements” — Brian Kernighan, Unix for Beginners.
This is the simplest way, somehow, an “effective” way to find your bugs in your code by using print functions to print all the significant variables and statements where you encountered your bug.
It is simple, but sometimes naive. We need to learn some advance techniques for better performance and efficiency. So that is why we need to learn debugging!
Logging
1 |
|
Logging is a great habit to make your debugging message neater! You can add several lines before the main functions to set the syntax of logging.
1 |
|
Then, you can output several log messages during your coding precess. Like this:
1 |
|
There are three types of logging messages.(Of course there are more!)
- Debugging: This level is used for detailed information, typically of interest only when diagnosing problems. It provides insights into the internal state of the application and is useful for developers during the debugging process.
- e.g. Logging variable values, function entry and exit points, or detailed flow of execution.
- INFO: This level is used to confirm that things are working as expected. It provides general information about the application’s progress and state without being overly verbose.
- e.g. Logging the start and completion of tasks, user actions, or significant milestones in the application.
- ERROR: This level indicates a more serious problem that prevented the program from performing a function. It is used to log exceptions or errors that occur during execution.
- e.g. Logging when a task fails due to an exception or when a critical operation cannot be completed.
- There does exist several other debugging messages! For instance, warning, critical, etc.
It’s ok if you don’t understand about threads in the codes shown above. You can just focus on the logging messages.
1 |
|
If you only want to output several error messages, you can just make more settings like this:
1 |
|
Of course, you can modify the logging message with different colors. You can modify the output colors of the terminal by using echo
command with different parameters.
For example, I have implemented this function in my ~/.zshrc
files.
1 |
|
You can copy this function into your dotfiles!
Debugger
Just simply output several logging messages isn’t enough! For example, you can not detect memory leak with printing all variables and messages. Thus, we need more specific tools to debugging!
Many programming languages come with some form of debugger. In Python this is the Python Debugger pdb
.
Here is a brief description of some of the commands pdb
supports:
- l(ist) - Displays 11 lines around the current line or continue the previous listing.
- s(tep) - Execute the current line, stop at the first possible occasion.
- n(ext) - Continue execution until the next line in the current function is reached or it returns.
- b(reak) - Set a breakpoint (depending on the argument provided).
- p(rint) - Evaluate the expression in the current context and print its value. There’s also pp to display using
pprint
instead. - r(eturn) - Continue execution until the current function returns.
- q(uit) - Quit the debugger.