Introduction to Computer Vision — The Machines That "See"

"If NLP taught machines to read, Computer Vision taught them to stop bumping into things." — probably me, in lecture form

You're coming out of the NLP module where we wrestled with tokenization, embeddings, and sequence quirks. Great! Now imagine we swap words for pixels, sequences for spatial grids, and sentences for scenes. Welcome to Computer Vision (CV): the branch of AI that gives machines visual common sense (or at least the illusion of it).

Why this matters now: many real-world AI systems are multimodal — they use text and images (think: image captioning, visual question answering, OCR plus context). So your NLP knowledge is not wasted; it’s a secret handshake that helps you understand how models combine language and vision.

---

What is Computer Vision, really?

Computer Vision is the field focused on enabling machines to interpret and understand images or video. That covers a wide range of goals:

• Recognize objects (Is that a cat?)
• Locate objects (Where is the cat?)
• Describe scenes (A cat on a red couch.)
• Track objects over time (Follow the cat as it jumps.)
• Reconstruct 3D shape from images (What does the cat look like from the other side?)

Think of CV as giving a machine eyes and a very literal brain: it sees arrays of pixel numbers and tries to map them to meaning.

---

Quick history (so we feel smart at parties)

• Pre-deep learning era: Hand-crafted features (SIFT, HOG), classical classifiers (SVMs). Engineers were feature-sculptors.
• 2012, AlexNet: Deep convolutional nets exploded performance in image classification. Suddenly, end-to-end learned features beat handcrafted ones. Deep learning got vision hot and mainstream.
• Now: Transformers, self-supervised learning, vision-language models — more flexible, more powerful, and sometimes more mysterious.

Why mention this? Because you'll often choose between classical and deep tools depending on data size, compute, and problem complexity.

---

Core concepts — the vocabulary you need

• Pixels: the basic units (numbers) of images. In NLP, pixels ~ tokens.
• Channels: RGB means 3 channels. Think of channels like feature dimensions for each token.
• Convolutional filters: small sliding windows that detect local patterns (edges, textures). They’re the visual analog of n-gram detectors.
• Feature maps: outputs after filters — like intermediate embeddings in NLP.
• Pooling: spatial summarization (downsampling) — like compressing context windows.
• Backbone: the main feature extractor (ResNet, ViT).
• Object detection vs classification: classification = "what is in the image?" detection = "where are the things and what are they?"

A table of parallels: NLP vs CV

NLP concept	CV analog	Role
Token	Pixel / Patch	Input unit
Embedding	Feature map	Learned representation
RNN / Transformer	ConvNet / ViT	Contextual modeling
Language model	Pretrained visual model	Transfer learning

---

Major tasks in computer vision (with friendly examples)

1. Image Classification — Is there a cat in the photo? (Single label)
2. Object Detection — Draw boxes around each cat and tell me which one is which. (Bounding boxes + classes)
3. Semantic Segmentation — Paint every pixel that belongs to 'cat' with the color purple. (Pixel-wise labels)
4. Instance Segmentation — Like segmentation but separate each cat instance with its own color.
5. Image Captioning / VQA (multimodal) — Generate a sentence describing the image / answer a question about it.
6. Pose Estimation / 3D reconstruction / Tracking — Fancier spatial reasoning tasks.

Ask yourself: "Which of these most resembles my problem?" That determines the loss, data annotation, and model choice.

---

Classical vs Deep — pick your fighter

• Classical methods (SIFT, HOG + SVM)
  • Pros: Interpretable, low data needs, cheap compute.
  • Cons: Limited performance on complex visual tasks.
• Deep learning methods (CNNs, ViTs)
  • Pros: State-of-the-art, learn features end-to-end, scale with data.
  • Cons: Data-hungry, compute-heavy, sometimes inscrutable.

Real projects often mix both: use deep models for heavy lifting, and classical heuristics for performance tweaks (post-processing, augmentations).

---

Practical pipeline — how a CV project usually flows

1. Define task & metrics (accuracy, mAP, IoU)
2. Collect & annotate data (this is the expensive part)
3. Choose model & pretraining (transfer learning is your friend)
4. Train & augment (cropping, color jitter, flips)
5. Evaluate, debug, iterate (visualize errors — humans are great at eye-balling failures)
6. Deploy & monitor (domain shift is real — lighting, camera changes, weird hats)

---

Challenges — because nothing in AI is easy

• Occlusion: objects hidden behind others.
• Lighting & weather: daylight vs night vs fog.
• Viewpoint variation: different angles confuse models.
• Domain shift: model trained on curated web photos fails on phone selfies.

Sound familiar? These are the visual cousins of NLP's ambiguity, polysemy, and domain transfer problems you already studied.

---

Quick code-like intuition (pseudocode)

# Pseudocode: image classification inference
image = load_image('photo.jpg')            # H x W x C array
image = preprocess(image)                  # resize, normalize
features = backbone(image)                 # conv layers -> feature map
logits = classifier(features)              # final dense layer
pred = softmax(logits)                     # probabilities
return argmax(pred)                        # predicted label

If that looked familiar, that's good — it's conceptually like passing tokens through an encoder and predicting labels.

---

Contrasting perspectives

Some researchers favor huge multimodal models (one giant model for everything), others argue for task-specific, efficient models tuned to constraints. Both approaches have merit — think of it as the "universal Swiss Army knife" vs the "customized chef's knife" debate.

---

Closing — key takeaways (aka remember these when your model cries)

• Computer Vision = turning pixels into meaning. It’s spatial, not sequential — but shares many ideas with NLP (embeddings, pretraining, transfer).
• Start with pretraining + transfer learning. Use existing backbones before building from scratch.
• Choose the right task and metric. Classification vs detection vs segmentation are different beasts.
• Data quality & diversity matter more than model magic. Models are only as good as what they see during training.

Final thought: Teaching a machine to see is like teaching a toddler — lots of examples, messy mistakes, and the occasional triumph where the machine correctly identifies a banana. Celebrate those tiny victories.

Ready to move from "what a sentence means" to "what a scene means"? Next up: Convolutional Neural Networks — where filters become feature detectives and receptive fields tell the model how much of the scene to care about.

---

Version notes: This builds on your NLP foundations (tokens → pixels, embeddings → feature maps) and sets the stage for diving into CNNs, ViTs, and multimodal models in the next lessons.