data/python_programming.txt

# Python Programming: Complete Guide from Basics to Advanced

## Introduction to Python

Python is a high-level, interpreted, and general-purpose programming language created by Guido van Rossum and first released in 1991. Known for its clear syntax and readability, Python has become one of the most popular programming languages in the world, especially in fields like data science, web development, automation, and artificial intelligence.

### Why Python?

**Advantages**:
- **Easy to Learn**: Simple, readable syntax resembling natural language
- **Versatile**: Used in web development, data science, AI, automation, and more
- **Large Community**: Extensive documentation and community support
- **Rich Ecosystem**: Thousands of libraries and frameworks
- **Cross-Platform**: Runs on Windows, macOS, Linux
- **Open Source**: Free to use and distribute
- **High-Level**: Automatic memory management
- **Interpreted**: No compilation needed, quick development cycle

**Use Cases**:
- Data Science and Machine Learning
- Web Development (Django, Flask)
- Automation and Scripting
- Scientific Computing
- Game Development
- Desktop Applications
- DevOps and System Administration

## Python Basics

### Installation and Setup

```python
# Check Python version
python --version  # or python3 --version

# Install packages using pip
pip install package_name

# Create virtual environment
python -m venv myenv

# Activate virtual environment
# Windows:
myenv\Scripts\activate
# macOS/Linux:
source myenv/bin/activate
```

### Variables and Data Types

```python
# Variables (no declaration needed)
name = "Alice"              # String
age = 25                    # Integer
height = 5.6                # Float
is_student = True           # Boolean
empty = None                # NoneType

# Type checking
print(type(name))           # <class 'str'>

# Type conversion
num_str = "42"
num_int = int(num_str)      # 42
num_float = float(num_str)  # 42.0
str_num = str(num_int)      # "42"
```

### Basic Data Structures

#### Lists (Mutable, Ordered)
```python
# Creating lists
fruits = ["apple", "banana", "cherry"]
numbers = [1, 2, 3, 4, 5]
mixed = [1, "two", 3.0, True]

# Accessing elements
first = fruits[0]           # "apple"
last = fruits[-1]           # "cherry"
slice = fruits[1:3]         # ["banana", "cherry"]

# Modifying lists
fruits.append("orange")     # Add to end
fruits.insert(1, "grape")   # Insert at index
fruits.remove("banana")     # Remove item
popped = fruits.pop()       # Remove and return last
fruits[0] = "pear"         # Change item

# List operations
length = len(fruits)
sorted_fruits = sorted(fruits)
fruits.reverse()
fruits.sort()

# List comprehension
squares = [x**2 for x in range(10)]
evens = [x for x in range(20) if x % 2 == 0]
```

#### Tuples (Immutable, Ordered)
```python
# Creating tuples
coordinates = (10, 20)
single = (42,)              # Single element tuple
empty = ()

# Accessing elements
x = coordinates[0]          # 10
y = coordinates[1]          # 20

# Unpacking
x, y = coordinates
a, *rest, b = (1, 2, 3, 4, 5)  # a=1, rest=[2,3,4], b=5

# Tuples are immutable
# coordinates[0] = 15       # This raises an error
```

#### Dictionaries (Key-Value Pairs)
```python
# Creating dictionaries
person = {
    "name": "Alice",
    "age": 25,
    "city": "New York"
}

# Accessing values
name = person["name"]
age = person.get("age")
city = person.get("country", "Unknown")  # Default value

# Modifying dictionaries
person["email"] = "alice@example.com"   # Add
person["age"] = 26                       # Update
del person["city"]                       # Delete
person.pop("email")                      # Remove and return

# Dictionary operations
keys = person.keys()
values = person.values()
items = person.items()

# Dictionary comprehension
squares = {x: x**2 for x in range(6)}
```

#### Sets (Unique, Unordered)
```python
# Creating sets
fruits = {"apple", "banana", "cherry"}
numbers = set([1, 2, 2, 3, 3, 3])  # {1, 2, 3}
empty = set()

# Set operations
fruits.add("orange")
fruits.remove("banana")
fruits.discard("grape")  # No error if doesn't exist

# Mathematical set operations
a = {1, 2, 3, 4}
b = {3, 4, 5, 6}
union = a | b              # {1, 2, 3, 4, 5, 6}
intersection = a & b       # {3, 4}
difference = a - b         # {1, 2}
symmetric_diff = a ^ b     # {1, 2, 5, 6}
```

### Control Flow

#### Conditional Statements
```python
# if-elif-else
age = 18
if age < 13:
    print("Child")
elif age < 20:
    print("Teenager")
else:
    print("Adult")

# Ternary operator
status = "Adult" if age >= 18 else "Minor"

# Multiple conditions
if age >= 18 and has_license:
    print("Can drive")

if name == "Alice" or name == "Bob":
    print("Welcome!")
```

#### Loops
```python
# For loops
for i in range(5):
    print(i)  # 0, 1, 2, 3, 4

for fruit in ["apple", "banana", "cherry"]:
    print(fruit)

for i, fruit in enumerate(fruits):
    print(f"{i}: {fruit}")

for key, value in person.items():
    print(f"{key}: {value}")

# While loops
count = 0
while count < 5:
    print(count)
    count += 1

# Break and continue
for i in range(10):
    if i == 3:
        continue  # Skip 3
    if i == 7:
        break     # Stop at 7
    print(i)

# Else clause (runs if loop completes normally)
for i in range(5):
    print(i)
else:
    print("Loop completed")
```

### Functions

```python
# Basic function
def greet(name):
    return f"Hello, {name}!"

result = greet("Alice")

# Default parameters
def power(base, exponent=2):
    return base ** exponent

print(power(3))      # 9
print(power(3, 3))   # 27

# Variable arguments
def sum_all(*args):
    return sum(args)

print(sum_all(1, 2, 3, 4))  # 10

# Keyword arguments
def person_info(**kwargs):
    for key, value in kwargs.items():
        print(f"{key}: {value}")

person_info(name="Alice", age=25, city="NYC")

# Lambda functions
square = lambda x: x**2
add = lambda x, y: x + y

# Higher-order functions
numbers = [1, 2, 3, 4, 5]
squared = list(map(lambda x: x**2, numbers))
evens = list(filter(lambda x: x % 2 == 0, numbers))

from functools import reduce
product = reduce(lambda x, y: x * y, numbers)
```

### Object-Oriented Programming

```python
# Class definition
class Dog:
    # Class variable
    species = "Canis familiaris"
    
    # Constructor
    def __init__(self, name, age):
        self.name = name      # Instance variable
        self.age = age
    
    # Instance method
    def bark(self):
        return f"{self.name} says Woof!"
    
    # String representation
    def __str__(self):
        return f"{self.name} is {self.age} years old"

# Creating instances
buddy = Dog("Buddy", 5)
miles = Dog("Miles", 3)

print(buddy.bark())      # Buddy says Woof!
print(miles)             # Miles is 3 years old

# Inheritance
class GoldenRetriever(Dog):
    def __init__(self, name, age, color):
        super().__init__(name, age)
        self.color = color
    
    def fetch(self):
        return f"{self.name} is fetching!"

max_dog = GoldenRetriever("Max", 2, "golden")

# Encapsulation
class BankAccount:
    def __init__(self, balance):
        self.__balance = balance  # Private attribute
    
    def deposit(self, amount):
        if amount > 0:
            self.__balance += amount
    
    def get_balance(self):
        return self.__balance

# Polymorphism
class Cat:
    def speak(self):
        return "Meow"

class Dog:
    def speak(self):
        return "Woof"

def animal_sound(animal):
    print(animal.speak())

animal_sound(Cat())  # Meow
animal_sound(Dog())  # Woof
```

## Advanced Python Concepts

### Decorators
```python
# Function decorator
def uppercase_decorator(func):
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        return result.upper()
    return wrapper

@uppercase_decorator
def greet(name):
    return f"hello, {name}"

print(greet("alice"))  # HELLO, ALICE

# Decorator with parameters
def repeat(times):
    def decorator(func):
        def wrapper(*args, **kwargs):
            for _ in range(times):
                result = func(*args, **kwargs)
            return result
        return wrapper
    return decorator

@repeat(3)
def say_hello():
    print("Hello!")
```

### Generators
```python
# Generator function
def count_up_to(n):
    count = 1
    while count <= n:
        yield count
        count += 1

counter = count_up_to(5)
for num in counter:
    print(num)  # 1, 2, 3, 4, 5

# Generator expression
squares = (x**2 for x in range(10))

# Fibonacci generator
def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib = fibonacci()
first_10 = [next(fib) for _ in range(10)]
```

### Context Managers
```python
# File handling with context manager
with open('file.txt', 'r') as file:
    content = file.read()
# File automatically closed

# Custom context manager
class DatabaseConnection:
    def __enter__(self):
        print("Opening connection")
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        print("Closing connection")
    
    def query(self, sql):
        print(f"Executing: {sql}")

with DatabaseConnection() as db:
    db.query("SELECT * FROM users")
```

### Exception Handling
```python
# Try-except
try:
    result = 10 / 0
except ZeroDivisionError:
    print("Cannot divide by zero!")
except Exception as e:
    print(f"An error occurred: {e}")
else:
    print("No errors!")
finally:
    print("Cleanup code")

# Raising exceptions
def divide(a, b):
    if b == 0:
        raise ValueError("Denominator cannot be zero")
    return a / b

# Custom exceptions
class InsufficientFundsError(Exception):
    pass

def withdraw(balance, amount):
    if amount > balance:
        raise InsufficientFundsError("Not enough funds")
    return balance - amount
```

### List, Dict, and Set Comprehensions
```python
# List comprehension
squares = [x**2 for x in range(10)]
evens = [x for x in range(20) if x % 2 == 0]
matrix = [[i*j for j in range(3)] for i in range(3)]

# Dictionary comprehension
square_dict = {x: x**2 for x in range(6)}
flipped = {v: k for k, v in person.items()}

# Set comprehension
unique_squares = {x**2 for x in [-2, -1, 0, 1, 2]}

# Nested comprehension
flattened = [num for row in matrix for num in row]
```

### Itertools and Collections
```python
from itertools import *
from collections import *

# Itertools
combinations_list = list(combinations([1, 2, 3], 2))
permutations_list = list(permutations([1, 2, 3], 2))
product_list = list(product([1, 2], ['a', 'b']))

# Infinite iterators
counter = count(start=10, step=2)
cycler = cycle(['A', 'B', 'C'])
repeater = repeat(10, times=3)

# Collections
counter = Counter(['a', 'b', 'a', 'c', 'b', 'a'])
# Counter({'a': 3, 'b': 2, 'c': 1})

default_dict = defaultdict(list)
default_dict['key'].append('value')

ordered_dict = OrderedDict()
ordered_dict['first'] = 1
ordered_dict['second'] = 2

named_tuple = namedtuple('Point', ['x', 'y'])
p = named_tuple(10, 20)
print(p.x, p.y)
```

## File Handling

```python
# Reading files
with open('file.txt', 'r') as f:
    content = f.read()           # Read entire file
    lines = f.readlines()        # Read lines into list
    
    for line in f:               # Iterate line by line
        print(line.strip())

# Writing files
with open('output.txt', 'w') as f:
    f.write("Hello, World!\n")
    f.writelines(["Line 1\n", "Line 2\n"])

# Appending
with open('output.txt', 'a') as f:
    f.write("Appended text\n")

# Working with CSV
import csv

with open('data.csv', 'r') as f:
    reader = csv.reader(f)
    for row in reader:
        print(row)

with open('output.csv', 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(['Name', 'Age'])
    writer.writerows([['Alice', 25], ['Bob', 30]])

# Working with JSON
import json

data = {'name': 'Alice', 'age': 25}

# Write JSON
with open('data.json', 'w') as f:
    json.dump(data, f, indent=2)

# Read JSON
with open('data.json', 'r') as f:
    data = json.load(f)
```

## Popular Python Libraries

### NumPy (Numerical Computing)
```python
import numpy as np

# Arrays
arr = np.array([1, 2, 3, 4, 5])
matrix = np.array([[1, 2, 3], [4, 5, 6]])

# Operations
print(arr * 2)
print(arr + 10)
print(np.mean(arr))
print(np.std(arr))

# Array creation
zeros = np.zeros((3, 3))
ones = np.ones((2, 4))
range_arr = np.arange(0, 10, 2)
linspace = np.linspace(0, 1, 5)
```

### Pandas (Data Manipulation)
```python
import pandas as pd

# DataFrames
df = pd.DataFrame({
    'name': ['Alice', 'Bob', 'Charlie'],
    'age': [25, 30, 35],
    'city': ['NYC', 'LA', 'Chicago']
})

# Operations
print(df.head())
print(df.describe())
print(df['age'].mean())

# Filtering
adults = df[df['age'] >= 30]

# Reading data
df = pd.read_csv('data.csv')
df = pd.read_excel('data.xlsx')
df = pd.read_json('data.json')
```

### Requests (HTTP Library)
```python
import requests

# GET request
response = requests.get('https://api.example.com/data')
data = response.json()

# POST request
payload = {'key': 'value'}
response = requests.post('https://api.example.com/submit', json=payload)

# With headers
headers = {'Authorization': 'Bearer token'}
response = requests.get('https://api.example.com/protected', headers=headers)
```

### Matplotlib (Visualization)
```python
import matplotlib.pyplot as plt

# Line plot
x = [1, 2, 3, 4, 5]
y = [1, 4, 9, 16, 25]
plt.plot(x, y)
plt.xlabel('X-axis')
plt.ylabel('Y-axis')
plt.title('Line Plot')
plt.show()

# Bar chart
categories = ['A', 'B', 'C']
values = [10, 20, 15]
plt.bar(categories, values)
plt.show()

# Scatter plot
plt.scatter(x, y)
plt.show()
```

## Best Practices

1. **PEP 8 Style Guide**: Follow Python's style conventions
2. **Meaningful Names**: Use descriptive variable and function names
3. **Documentation**: Write docstrings for functions and classes
4. **Error Handling**: Use try-except blocks appropriately
5. **Virtual Environments**: Isolate project dependencies
6. **Type Hints**: Use type annotations for clarity
7. **List Comprehensions**: Use when appropriate for readability
8. **Context Managers**: Use 'with' statements for resource management
9. **DRY Principle**: Don't Repeat Yourself
10. **Testing**: Write unit tests for your code

## Conclusion

Python's simplicity, readability, and extensive ecosystem make it an ideal language for beginners and experts alike. Whether you're building web applications, analyzing data, training machine learning models, or automating tasks, Python provides the tools and libraries to get the job done efficiently.

Continuous learning and practice are key to mastering Python. Explore different libraries, contribute to open-source projects, and build your own applications to deepen your understanding and skills.