Jhear

High-Performance Microservices with @hazeljs/grpc

Muhammad Arslan — Sun, 22 Feb 2026 16:52:13 GMT

I recently added gRPC support to HazelJS — a TypeScript-first Node.js framework. The new @hazeljs/grpc package lets you build gRPC servers with a decorator-based API, full dependency injection, and the familiar module pattern.

Why gRPC?

Microservices need efficient, type-safe communication. gRPC gives you high-performance RPC over HTTP/2 with Protocol Buffers for serialization. Instead of wiring proto loading, service registration, and handlers manually, the HazelJS gRPC module handles it with decorators and DI.

Features

Decorator-based handlers — Use @GrpcMethod('ServiceName', 'MethodName') to declare RPC handlers
Full DI integration — Controllers are resolved from the HazelJS container; inject services and repositories
Proto loading — Load .proto files at runtime with configurable options
Discovery compatible — Works with the HazelJS Discovery package for service registration (protocol: 'grpc')

Installation

npm install @hazeljs/grpc

npm: @hazeljs/grpc

Quick Start

Define your service in a .proto file:

syntax = "proto3";
package hero;

service HeroService {
  rpc FindOne (HeroById) returns (Hero);
}

message HeroById { int32 id = 1; }
message Hero { int32 id = 1; string name = 2; }

Create a controller with @GrpcMethod:

import { Injectable, HazelModule } from '@hazeljs/core';
import { GrpcMethod, GrpcModule } from '@hazeljs/grpc';
import { join } from 'path';

@Injectable()
export class HeroGrpcController {
  @GrpcMethod('HeroService', 'FindOne')
  findOne(data: { id: number }) {
    return { id: data.id, name: 'Hero' };
  }
}

@HazelModule({
  imports: [
    GrpcModule.forRoot({
      protoPath: join(__dirname, 'hero.proto'),
      package: 'hero',
      url: '0.0.0.0:50051',
    }),
  ],
  providers: [HeroGrpcController],
})
export class AppModule {}

import { HazelApp } from '@hazeljs/core';
import { GrpcModule, GrpcServer } from '@hazeljs/grpc';
import { Container } from '@hazeljs/core';

const app = new HazelApp(AppModule);
GrpcModule.registerHandlersFromProviders([HeroGrpcController]);
await app.listen(3000);

const grpcServer = Container.getInstance().resolve(GrpcServer);
await grpcServer.start();

Concrete Example: Product Catalog Service

Here's a complete example — a product catalog gRPC service that fetches products from a repository and supports both lookup by ID and listing with pagination.

product.proto

syntax = "proto3";
package catalog;

service ProductService {
  rpc GetProduct (GetProductRequest) returns (Product);
  rpc ListProducts (ListProductsRequest) returns (ListProductsResponse);
}

message GetProductRequest {
  string id = 1;
}

message Product {
  string id = 1;
  string name = 2;
  string description = 3;
  double price = 4;
}

message ListProductsRequest {
  int32 page = 1;
  int32 pageSize = 2;
}

message ListProductsResponse {
  repeated Product products = 1;
  int32 total = 2;
}

product.repository.ts — In-memory store (replace with Prisma/DB in production):

import { Injectable } from '@hazeljs/core';

@Injectable()
export class ProductRepository {
  private products = [
    { id: '1', name: 'Widget A', description: 'A useful widget', price: 9.99 },
    { id: '2', name: 'Widget B', description: 'Another widget', price: 14.99 },
    { id: '3', name: 'Gadget X', description: 'A fancy gadget', price: 29.99 },
  ];

  findById(id: string) {
    return this.products.find((p) => p.id === id) ?? null;
  }

  findAll(page: number, pageSize: number) {
    const start = (page - 1) * pageSize;
    const items = this.products.slice(start, start + pageSize);
    return { products: items, total: this.products.length };
  }
}

product.grpc-controller.ts — gRPC controller with injected repository:

import { Injectable } from '@hazeljs/core';
import { GrpcMethod } from '@hazeljs/grpc';
import { ProductRepository } from './product.repository';

@Injectable()
export class ProductGrpcController {
  constructor(private readonly productRepo: ProductRepository) {}

  @GrpcMethod('ProductService', 'GetProduct')
  async getProduct(data: { id: string }) {
    const product = this.productRepo.findById(data.id);
    if (!product) {
      throw new Error(`Product ${data.id} not found`);
    }
    return product;
  }

  @GrpcMethod('ProductService', 'ListProducts')
  async listProducts(data: { page?: number; pageSize?: number }) {
    const page = data.page ?? 1;
    const pageSize = data.pageSize ?? 10;
    return this.productRepo.findAll(page, pageSize);
  }
}

app.module.ts

import { HazelModule } from '@hazeljs/core';
import { GrpcModule } from '@hazeljs/grpc';
import { join } from 'path';
import { ProductGrpcController } from './product.grpc-controller';
import { ProductRepository } from './product.repository';

@HazelModule({
  imports: [
    GrpcModule.forRoot({
      protoPath: join(__dirname, 'product.proto'),
      package: 'catalog',
      url: '0.0.0.0:50051',
    }),
  ],
  providers: [ProductRepository, ProductGrpcController],
})
export class AppModule {}

main.ts

import { HazelApp } from '@hazeljs/core';
import { GrpcModule, GrpcServer } from '@hazeljs/grpc';
import { Container } from '@hazeljs/core';
import { AppModule } from './app.module';
import { ProductGrpcController } from './product.grpc-controller';

async function bootstrap() {
  const app = new HazelApp(AppModule);
  GrpcModule.registerHandlersFromProviders([ProductGrpcController]);
  await app.listen(3000);

  const grpcServer = Container.getInstance().resolve(GrpcServer);
  await grpcServer.start();

  console.log('HTTP on :3000, gRPC on :50051');
}
bootstrap();

Test with grpcurl:

# Get a product by ID
grpcurl -plaintext -d '{"id":"1"}' localhost:50051 catalog.ProductService/GetProduct

# List products with pagination
grpcurl -plaintext -d '{"page":1,"pageSize":2}' localhost:50051 catalog.ProductService/ListProducts

Learn More

Documentation: HazelJS gRPC Package Docs
npm: @hazeljs/grpc
HazelJS Blog: High-Performance Microservices with @hazeljs/grpc

AI-Powered Ops: Jira, Slack, and Incident Triage with @hazeljs/ops-agent

Muhammad Arslan — Fri, 13 Feb 2026 13:28:49 GMT

Introducing @hazeljs/ops-agent — an AI-powered DevOps assistant that creates Jira tickets, posts to Slack, and coordinates incidents through natural language. Built on the HazelJS Agent Runtime with built-in Jira and Slack adapters.

Why an Ops Agent?

SRE and DevOps workflows often involve multiple tools: create a ticket in Jira Cloud, notify the team in Slack, add a comment, look up runbooks. Manually switching between these tools during an incident is slow and error-prone. The Ops Agent lets you say things like:

"Create a Jira ticket in PROJ for database connection pool exhaustion and post to #incidents"
"Add a comment to PROJ-123 with the latest status and notify #sre"

The agent parses your intent, calls the right tools with the right parameters, and confirms what it did. Sensitive actions like creating tickets or posting to Slack can require approval, so you stay in control.

Installation

npm install @hazeljs/ops-agent @hazeljs/ai @hazeljs/agent @hazeljs/core @hazeljs/rag

Quick Start

Configure the Jira and Slack tools, create the runtime, and run the agent:

import { AIEnhancedService } from '@hazeljs/ai';
import { createOpsRuntime, runOpsAgent, createJiraTool, createSlackTool } from '@hazeljs/ops-agent';

const jiraTool = createJiraTool();
const slackTool = createSlackTool();

const runtime = createOpsRuntime({
  aiService: new AIEnhancedService(),
  tools: { jira: jiraTool, slack: slackTool },
  model: 'gpt-4',
});

const result = await runOpsAgent(runtime, {
  input: 'Create a Jira ticket in PROJ for DB pool exhaustion and post to #incidents.',
  sessionId: `ops-${Date.now()}`,
});

console.log(result.response);
console.log(`Completed in ${result.steps} steps (${result.duration}ms)`);

Configuration

For real Jira and Slack integration, set these environment variables:

OPENAI_API_KEY — Required for the LLM
JIRA_HOST, JIRA_EMAIL, JIRA_API_TOKEN — For Jira Cloud REST API v3
SLACK_BOT_TOKEN — For Slack Web API (e.g. chat.postMessage)

Without Jira or Slack credentials, the adapters return placeholder responses so you can develop and test locally. Set the env vars when you're ready for production.

Tools

The Ops Agent exposes four tools to the LLM:

create_jira_ticket — Create issues with project, summary, description, and optional labels (requires approval)
add_jira_comment — Add comments to existing issues
get_jira_ticket — Fetch issue details by key
post_to_slack — Post messages to channels, optionally in threads (requires approval)

RAG and Persistent Memory

Optionally pass a ragService so the agent can search runbooks and docs before suggesting actions. Use a custom memoryManager (e.g. Redis) for persistent conversation history across sessions — see the HazelJS RAG package for memory and retrieval options.

Learn More

Ops Agent Package Documentation — Full API reference, custom tools, and advanced configuration
Production Agent Runtime — Stateful AI agents with tools and memory
Agentic RAG — Self-improving agents with RAG
GitHub: hazeljs/hazeljs — Source code and examples including example/src/ops/ops-agent-example.ts

@hazeljs/ops-agent is part of the HazelJS framework — a modern, TypeScript-first Node.js framework for building scalable server-side applications.

Introducing HazelJS: The AI-Native TypeScript Framework

Muhammad Arslan — Sat, 07 Feb 2026 09:27:26 GMT

I'm excited to announce the official launch of HazelJS - the first truly AI-native TypeScript framework designed to accelerate backend development with built-in AI capabilities, autonomous agents, and enterprise-grade features out of the box.

What is HazelJS?

HazelJS is a modern, AI-first TypeScript framework that combines the best practices of dependency injection, modular architecture, and production-ready features into a single, cohesive ecosystem. Built with AI at its core, it provides everything you need to build intelligent, scalable, and maintainable backend applications powered by cutting-edge AI technologies.

Why We Built HazelJS: The AI-Native Approach

After years of building backend systems, we noticed a seismic shift in how applications are being built. AI is no longer a feature - it's becoming the foundation. Developers are struggling with:

Fragmented AI integrations: Different SDKs for each AI provider with inconsistent APIs
Complex agent architectures: Building autonomous agents requires orchestrating LLMs, tools, memory, and state management
RAG implementation challenges: Vector databases, embeddings, and semantic search are hard to get right
Lack of AI-native frameworks: Traditional frameworks weren't designed for the AI-first era

We created HazelJS to solve these problems - providing the first truly AI-native framework where AI capabilities are first-class citizens, not afterthoughts. Build intelligent applications with the same ease as traditional CRUD apps.

Key Features

🎯 Core Framework

Dependency Injection: Powerful IoC container with decorator-based configuration
Modular Architecture: Build applications with reusable, testable modules
Type Safety: Full TypeScript support with comprehensive type definitions
Middleware & Guards: Flexible request/response pipeline with authentication and authorization

🤖 AI-Native Integration (The Heart of HazelJS)

One unified API for all AI providers - switch between models without changing your code:

OpenAI: GPT-4, GPT-4 Turbo, GPT-3.5 - Industry-leading language models
Anthropic: Claude 3.5 Sonnet, Claude 3 Opus - Advanced reasoning and analysis
Cohere: Command, Embed, Rerank - Enterprise-grade NLP
Google Gemini: Multimodal AI capabilities

// Same code works with any provider!
const response = await ai.complete({
  provider: 'openai', // or 'anthropic', 'cohere', 'gemini'
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'Explain quantum computing' }]
});

Why this matters: Build once, deploy anywhere. Test with different models, optimize costs, avoid vendor lock-in.

🔐 Authentication & Security

JWT-based authentication out of the box
Role-based access control (RBAC)
OAuth 2.0 integration
Security best practices built-in

⚡ Performance & Scalability

Multi-tier Caching: Memory, Redis, and CDN support
WebSocket & SSE: Real-time communication with room management
Serverless Ready: Deploy to AWS Lambda or Google Cloud Functions
Service Discovery: Microservices architecture support

🧠 AI-Native Advanced Capabilities

Autonomous AI Agents 🤖

Build intelligent agents that can think, plan, and execute tasks autonomously:

Tool Execution: Agents can call functions and APIs to interact with the real world
Memory Management: Persistent context and conversation history
State Management: Track agent execution across multiple steps
Approval Workflows: Human-in-the-loop for sensitive operations

const agent = new Agent({
  name: 'DataAnalyst',
  description: 'Analyzes data and generates insights',
  tools: [queryDatabase, generateChart, sendEmail],
  llm: { provider: 'openai', model: 'gpt-4' }
});

const result = await agent.execute('Analyze last month sales and email report');

RAG (Retrieval-Augmented Generation) 🔍

Production-ready semantic search and knowledge retrieval:

Vector Embeddings: Generate embeddings with any provider
Semantic Search: Find relevant context using cosine similarity
Multiple Vector Stores: Support for Pinecone, Weaviate, and more
Automatic Chunking: Smart document splitting for optimal retrieval

// Add knowledge to your AI
await rag.addDocuments([
  { content: 'Product documentation...', metadata: { source: 'docs' } }
]);

// Query with semantic search
const context = await rag.search('How do I configure caching?', { topK: 3 });

// Use in AI completions
const answer = await ai.complete({
  provider: 'openai',
  messages: [
    { role: 'system', content: `Context: ${context}` },
    { role: 'user', content: 'How do I configure caching?' }
  ]
});

Traditional Backend Features

Cron Jobs: Scheduled task execution with timezone support
Prisma Integration: Type-safe database access with repository pattern

Getting Started

Getting started with HazelJS is simple:

# Install the CLI
npm install -g @hazeljs/cli

# Create a new project
hazel new my-app

# Start building!
cd my-app
npm run dev

Visit the official documentation for comprehensive guides and examples.

AI-Native Use Cases

HazelJS excels at building modern AI-powered applications:

🎯 Intelligent Chatbots & Virtual Assistants

Build conversational AI with context awareness, tool calling, and multi-turn conversations. Perfect for customer support, internal tools, or AI companions.

📊 AI-Powered Data Analysis

Create autonomous agents that can query databases, analyze data, generate visualizations, and provide insights - all through natural language.

📚 Knowledge Base & Documentation AI

Implement RAG systems that can answer questions about your documentation, codebase, or internal knowledge base with semantic search.

🔧 AI Workflow Automation

Build agents that can orchestrate complex workflows, integrate with external APIs, and make decisions based on business logic.

💬 Content Generation & Summarization

Generate blog posts, summaries, translations, or any text-based content with support for multiple AI providers and streaming responses.

🔍 Semantic Search & Recommendations

Implement vector-based search for products, content, or documents with intelligent ranking and personalization.

Real-World Example

Here's a simple example of building an AI-powered API endpoint with HazelJS:

import { Controller, Post, Body, Injectable } from '@hazeljs/core';
import { AIService } from '@hazeljs/ai';

@Injectable()
export class ChatService {
  constructor(private ai: AIService) {}

  async chat(message: string) {
    return await this.ai.complete({
      provider: 'openai',
      model: 'gpt-4',
      messages: [{ role: 'user', content: message }],
    });
  }
}

@Controller('/api/chat')
export class ChatController {
  constructor(private chatService: ChatService) {}

  @Post()
  async handleChat(@Body() body: { message: string }) {
    return await this.chatService.chat(body.message);
  }
}

That's it! You have a fully functional AI-powered API with proper dependency injection, type safety, and error handling.

The AI-Native Architecture Advantage

What makes HazelJS truly AI-native? It's not just about having AI integrations - it's about designing the entire framework around AI-first principles:

Unified AI Abstraction Layer

Instead of learning different SDKs for each AI provider, HazelJS provides a single, consistent API. Switch between OpenAI, Anthropic, Cohere, or Gemini with a single configuration change.

Built-in Agent Orchestration

Most frameworks treat agents as an afterthought. HazelJS includes a production-ready agent runtime with:

Tool registration and execution
Conversation memory and state management
Multi-step reasoning and planning
Error handling and retry logic
Human approval workflows

First-Class RAG Support

Vector search and semantic retrieval are core primitives, not third-party plugins. Generate embeddings, store vectors, and perform semantic search with the same ease as querying a database.

Streaming by Default

AI responses can be slow. HazelJS supports streaming responses out of the box, providing real-time feedback to users as the AI generates content.

Cost & Performance Optimization

Intelligent caching: Cache AI responses to reduce API costs
Provider fallbacks: Automatically switch providers if one fails
Token counting: Track usage and optimize prompts
Rate limiting: Prevent API quota exhaustion

Production-Ready Features

HazelJS isn't just another framework - it's built for production from day one:

✅ 1,129+ passing tests across all packages
✅ Automated CI/CD with GitHub Actions
✅ Comprehensive documentation with real-world examples
✅ Active development with regular updates
✅ TypeScript-first with full type safety
✅ Monorepo architecture for easy maintenance

Package Ecosystem

HazelJS is built as a modular monorepo with 14 specialized packages:

@hazeljs/core - Core framework and dependency injection
@hazeljs/ai - AI provider integrations
@hazeljs/agent - AI agent runtime with tools and memory
@hazeljs/auth - Authentication and authorization
@hazeljs/cache - Multi-tier caching system
@hazeljs/websocket - Real-time communication
@hazeljs/serverless - Serverless deployment adapters
@hazeljs/rag - Vector search and embeddings
@hazeljs/prisma - Database ORM integration
@hazeljs/cron - Scheduled task execution
And more!

Each package can be used independently or together as a complete framework.

Open Source & Community

HazelJS is fully open source and available on GitHub. We believe in building in public and welcome contributions from the community.

Get Involved

⭐ Star us on GitHub
📖 Read the documentation
💬 Join our Discord community
🐛 Report issues or request features
🤝 Contribute to the project

What's Next?

We're just getting started! Our roadmap includes:

GraphQL support
gRPC integration
Enhanced monitoring and observability
More AI provider integrations
Advanced agent capabilities
Performance optimizations

Try It Today

Ready to accelerate your backend development? Install HazelJS today:

npm install @hazeljs/core@beta

Visit hazeljs.com to explore the full documentation, examples, and guides.

About the Author

I'm Muhammad Arslan, a software engineer passionate about building tools that make developers' lives easier. HazelJS is the culmination of years of experience building production systems and learning what works at scale.

Follow the project on GitHub and visit hazeljs.com to stay updated with the latest developments.

Links:

🌐 Website: hazeljs.com
📚 Documentation: hazeljs.com/docs
💻 GitHub: github.com/hazel-js/hazeljs
📦 NPM: @hazeljs

REST vs gRPC vs GraphQL — A Complete Guide With Java Examples

Muhammad Arslan — Tue, 09 Dec 2025 18:11:06 GMT

APIs power every modern application — mobile apps, microservices, dashboards, IoT devices, and more. And today, developers have three major API technologies to choose from:

REST (HTTP/JSON)
gRPC (HTTP/2 + Protocol Buffers)
GraphQL (single endpoint + flexible queries)

Each has its strengths, weaknesses, and ideal use cases.

🌐 1. REST — The Traditional Standard

REST is the most widely used API style, built around:

Resources (/users, /orders)
HTTP verbs (GET, POST, PUT, DELETE)
JSON as the default format

✔ Strengths

Easy to understand
Browser-friendly
Great tooling
Good for public APIs

❌ Weaknesses

Over-fetching
Under-fetching
JSON overhead
HTTP/1.1 latency

Java Example

@RestController
@RequestMapping("/users")
public class UserController {

    @GetMapping("/{id}")
    public User getUser(@PathVariable int id) {
        return new User(id, "John Doe");
    }
}

⚡ 2. gRPC — High-Performance, Low-Latency APIs

gRPC uses:

HTTP/2
Protocol Buffers
Generated client/server code
Streaming

✔ Strengths

Extremely fast
Small binary payloads
Streaming support
Strong typing

❌ Weaknesses

Not browser-native
Harder to debug binary payloads

gRPC Java Example

user.proto:

syntax = "proto3";

service UserService {
  rpc GetUser(GetUserRequest) returns (GetUserResponse);
}

message GetUserRequest {
  int32 id = 1;
}

message GetUserResponse {
  int32 id = 1;
  string name = 2;
}

Server:

public class UserServiceImpl extends UserServiceGrpc.UserServiceImplBase {

    @Override
    public void getUser(GetUserRequest request, StreamObserver responseObserver) {

        GetUserResponse response = GetUserResponse.newBuilder()
                .setId(request.getId())
                .setName("John Doe")
                .build();

        responseObserver.onNext(response);
        responseObserver.onCompleted();
    }
}

🔍 3. GraphQL — Flexible Queries for Frontend Apps

GraphQL allows clients to request exactly the data they need.

✔ Strengths

Avoids over-fetching
Strong schema
Great for mobile & dashboards

❌ Weaknesses

Harder caching
Over-fetching risks in resolvers
Performance overhead in large graphs

Java Example (Spring GraphQL)

Schema:

type User {
  id: Int
  name: String
}

type Query {
  user(id: Int!): User
}

Resolver:

@Component
public class UserQueryResolver implements GraphQLQueryResolver {
    public User user(int id) {
        return new User(id, "John Doe");
    }
}

🏎 Performance Comparison

Feature	REST	gRPC	GraphQL
Transport	HTTP/1.1	HTTP/2	HTTP/1.1
Format	JSON	ProtoBuf (binary)	JSON
Speed	Medium	Fastest	Medium
Streaming	❌ No	✔ Yes	⚠ Partial
Browser support	✔ Excellent	⚠ Needs gRPC-Web	✔ Excellent
Typing	Weak	Strong	Strong
Ideal for	Public APIs	Microservices	Frontend apps

🎯 When to Choose What?

✔ Use REST when:

Building public APIs
Browser-based clients
Simplicity matters

✔ Use gRPC when:

You need performance
Microservices communicate internally
You need streaming

✔ Use GraphQL when:

Frontend needs flexible queries
You want to avoid over-fetching
Mobile apps require optimized data loads

🧠 Final Summary

REST → Simple, universal
gRPC → Fast, efficient, best for microservices
GraphQL → Flexible, client-driven

Each excels in different scenarios.

Understanding gRPC: How It Works and Why It’s So Performant

Muhammad Arslan — Mon, 08 Dec 2025 08:49:22 GMT

Modern distributed applications demand high-performance communication between services. While REST over HTTP/1.1 has served us well, microservices, streaming systems, IoT, and high-throughput backends now require more efficient, low-latency alternatives.

Enter gRPC — Google's high-performance RPC framework that combines Protocol Buffers, HTTP/2, and code-generated clients to deliver unmatched speed.

In this article, we’ll explore:

What gRPC is
How gRPC works internally
Why gRPC is incredibly fast
gRPC streaming types
A practical Java example (client + server)
When to use gRPC vs REST

🌐 What Is gRPC?

gRPC is a Remote Procedure Call (RPC) framework that allows you to call methods on a remote server as if they were local functions.

It is built on three core pillars:

Protocol Buffers (binary serialization)
HTTP/2 transport
Strongly-typed auto-generated client/server code

This enables efficient communication between services written in different languages (Java, Go, Python, Node.js, C#, Rust, etc.).

🧠 How gRPC Works

1. Define your API using Protocol Buffers

You write a .proto file defining:

Services
RPC methods
Request/response message types

Example: user.proto

syntax = "proto3";

package user;
option java_multiple_files = true;

service UserService {
  rpc GetUser(GetUserRequest) returns (GetUserResponse);
}

message GetUserRequest {
  int32 id = 1;
}

message GetUserResponse {
  int32 id = 1;
  string name = 2;
}

Compile the proto file:

protoc --java_out=./generated --grpc-java_out=./generated user.proto

This generates:

Java classes for your messages
A service interface for the server
A client stub for the client

🚀 Why gRPC Is So Performant

Let’s break down the magic behind gRPC’s performance.

1. HTTP/2 Instead of HTTP/1.1

HTTP/2 provides:

✔ Multiplexing

Multiple requests sent at the same time over one TCP connection.

✔ Header Compression (HPACK)

Reduces repeated metadata, saving bandwidth.

✔ Server Push

Great for real-time streaming.

2. Protocol Buffers Are Extremely Efficient

Format	Serialization	Size	CPU Cost
JSON	Text	Large	High
Protocol Buffers	Binary	Very small	Very low

ProtoBuf messages shrink payloads dramatically—often by 70–90%—and encode/decode extremely fast.

3. Auto-generated Code = Speed

gRPC generates:

Server base classes
Strongly typed client stubs
Efficient serialization logic

No slow reflection or manual parsing.
Just fast, compiled logic.

4. Streaming Is a First-Class Feature

gRPC supports:

Type	Description
Unary	1 request → 1 response
Server Streaming	1 request → stream of responses
Client Streaming	Stream of requests → 1 response
Bidirectional Streaming	Both sides stream independently

Combined with HTTP/2, these are extremely lightweight and low latency.

💻 Building a gRPC Service in Java

Let’s build a full working example.

1️⃣ Implement the gRPC Server

`UserServiceImpl.java`

package user;

import io.grpc.stub.StreamObserver;

public class UserServiceImpl extends UserServiceGrpc.UserServiceImplBase {

    @Override
    public void getUser(GetUserRequest request, StreamObserver responseObserver) {
        int id = request.getId();

        GetUserResponse response = GetUserResponse.newBuilder()
                .setId(id)
                .setName("John Doe")
                .build();

        responseObserver.onNext(response);
        responseObserver.onCompleted();
    }
}

2️⃣ Start the gRPC Server

`GrpcServer.java`

import io.grpc.Server;
import io.grpc.ServerBuilder;

public class GrpcServer {

    public static void main(String[] args) throws Exception {
        Server server = ServerBuilder
                .forPort(9090)
                .addService(new UserServiceImpl())
                .build();

        System.out.println("🚀 gRPC Server running on port 9090");
        server.start();
        server.awaitTermination();
    }
}

Run it:

java GrpcServer

3️⃣ Implement the gRPC Client

`GrpcClient.java`

import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import user.GetUserRequest;
import user.UserServiceGrpc;

public class GrpcClient {

    public static void main(String[] args) {

        ManagedChannel channel = ManagedChannelBuilder
                .forAddress("localhost", 9090)
                .usePlaintext()
                .build();

        UserServiceGrpc.UserServiceBlockingStub client =
                UserServiceGrpc.newBlockingStub(channel);

        GetUserRequest request = GetUserRequest.newBuilder()
                .setId(1)
                .build();

        var response = client.getUser(request);

        System.out.println("User name: " + response.getName());
        channel.shutdown();
    }
}

✔ Example Output

User name: John Doe

📌 When Should You Use gRPC?

High-performance microservices
Real-time streaming (chat, telemetry, IoT)
Low-latency systems (trading, gaming)
Polyglot architectures
Internal service communication

❌ When Not to Use gRPC

Public browser APIs (unless using gRPC-Web)
When debugging readability matters
When simple JSON over HTTP is enough

🏁 Conclusion

gRPC is incredibly fast because it brings together:

HTTP/2 (multiplexing, header compression, low latency)
Protocol Buffers (compact, binary, efficient)
Generated stubs (optimized, type-safe code)
Built-in streaming capabilities

With just a .proto file, you can generate a fully working Java client and server that communicate efficiently at scale.

If your system needs speed, scalability, and real-time communication, gRPC is one of the best tools available today.

Part 2 — Python for Data Science: Essential Tools and Idioms

Muhammad Arslan — Thu, 04 Dec 2025 14:01:58 GMT

This post gets you productive with the core Python stack for data work: pandas, NumPy, and Jupyter. You will create a clean workspace, learn idiomatic patterns, and run a small example on provided sample data. If Part 1 was about “what” and “why,” this part is about “how do I start doing it with the right habits.”

Why Python for Data Science?

Huge ecosystem: pandas for tabular data, NumPy for arrays, scikit-learn for ML, Matplotlib/Seaborn/Plotly for viz.
Fast path from prototype to production: the same language can power notebooks, APIs (FastAPI), and pipelines (Airflow/Prefect).
Community and packages: almost every data source (databases, cloud storage, APIs) has a Python client.
Ergonomics: readable syntax, rich notebooks, and plenty of learning resources.

Core Tools at a Glance

Tool	Purpose	Notes
pandas	Tabular data (DataFrames), joins, groupby, IO	Think “Excel + SQL + Python”
NumPy	Fast n-dimensional arrays and math	Underpins pandas; great for vectorized ops
JupyterLab	Interactive notebooks for exploration	Mix code, text, and charts
Matplotlib/Seaborn	Plotting basics and statistical visuals	Seaborn builds on Matplotlib for nicer defaults
VS Code (optional)	IDE for refactoring and debugging	Use with Python + Jupyter extensions

Learning Goals

Set up a reproducible Python environment and Jupyter workspace (so your notebook runs the same next week).
Load and inspect tabular data with pandas; use NumPy for fast array math.
Apply clean code patterns: tidy data, assign, pipe, query, and explicit dtypes.
Keep a project folder organized so future you (and teammates) can debug quickly.
Know what “good enough” exploration looks like before you jump to modeling.

Setup: Environment and Dependencies

Install Python 3.11+ (or your team’s standard). Consistency avoids “works on my machine.”
Create a virtual environment and install essentials (keeps dependencies isolated per project):

python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install pandas numpy jupyterlab matplotlib seaborn

Shortcut: run make install (creates .venv and installs from requirements.txt), then make notebook to launch JupyterLab.

Launch JupyterLab in this repo for notebooks:

jupyter lab

Optional VS Code setup: install the Python and Jupyter extensions; set the interpreter to .venv/bin/python.
Alternative: if you prefer Conda, create an environment with conda create -n ds python=3.11 pandas numpy jupyterlab seaborn.

Quick Data Check: pandas + NumPy in Action

Sample files: part2/orders.csv (orders with product, quantity, price) and part2/customers.csv (customer segment and country). This tiny example mirrors a common starter task: join transactions to customer attributes, compute revenue, and summarize by segment.

import pandas as pd
import numpy as np

orders = pd.read_csv("part2/orders.csv", parse_dates=["order_date"])
customers = pd.read_csv("part2/customers.csv")

# Tidy types for memory and consistency
orders["product"] = orders["product"].astype("category")
customers["segment"] = customers["segment"].astype("category")

# Basic quality checks (fast assertions catch bad inputs early)
assert orders["quantity"].ge(0).all()
assert orders["price"].ge(0).all()

# Revenue per order
orders = orders.assign(order_revenue=lambda d: d["quantity"] * d["price"])

# Join customer info (left join preserves all orders)
orders = orders.merge(customers, on="customer_id", how="left")

# Segment-level summary
summary = (
    orders.groupby("segment")
          .agg(
              orders=("order_id", "count"),
              revenue=("order_revenue", "sum"),
              avg_order_value=("order_revenue", "mean"),
          )
          .sort_values("revenue", ascending=False)
)
print(summary)

# NumPy example: standardize revenue for quick z-scores
orders["revenue_z"] = (orders["order_revenue"] - orders["order_revenue"].mean()) / orders["order_revenue"].std(ddof=0)

What you just did:

Loaded CSVs with explicit date parsing and dtypes (prevents surprises later).
Added a computed column via assign to keep the transformation readable.
Joined customer attributes to transactions with a left join (common pattern).
Summarized by segment to answer “which customer type drives revenue?”
Used NumPy to add a quick z-score—handy for outlier checks or bucketing.

Expected summary output (with the sample data):

            orders  revenue  avg_order_value
segment
Enterprise        4   651.00          162.75
SMB               5   385.50           77.10
Consumer          1    66.00           66.00

Use this as a quick sense check: values are positive, orders count matches the CSV, and Enterprise drives the most revenue.

Companion Notebook

Path: part2/notebooks/part2-example.ipynb.
Run with the Makefile: make install (first time) then make notebook and open the notebook.
Without Makefile: jupyter lab part2/notebooks/part2-example.ipynb (use your activated environment).
Check kaggle here

Idiomatic pandas Patterns

assign: Add columns without breaking method chains.
pipe: Encapsulate reusable transformations and keep chains readable.
query: Express simple filters with readable expressions.
Explicit dtypes: use astype and to_datetime to avoid silent conversions.
Small helpers: prefer value_counts(normalize=True) for quick proportions.

Example using pipe and query:

def add_order_revenue(df):
    return df.assign(order_revenue=lambda d: d["quantity"] * d["price"])

(orders
 .pipe(add_order_revenue)
 .query("order_revenue > 200")
 .groupby("product")["order_revenue"]
 .mean()
)

Notebook Hygiene and Habits

Start every notebook with imports, configuration, and a short Markdown cell stating the question you are answering.
Pin a random seed for reproducibility when sampling or modeling.
Keep side effects contained: write outputs to a data/ or reports/ folder, not your repo root.
Restart-and-run-all before sharing; if it fails, fix it before committing.
When a notebook grows too large, move reusable code into src/ functions and re-import—treat notebooks as experiments, not long-term code storage.

Common Pitfalls to Avoid Early

Silent type coercion: always inspect df.dtypes after loading; parse dates explicitly.
Chained indexing (df[df["x"] > 0]["y"] = ...) can create copies—use .loc and assign instead.
Skipping data checks: use quick assertions for non-negativity, allowed categories, and unique keys.
Mixing raw and cleaned data: keep a clear path (raw → interim/clean → features) with filenames that show the stage.

Workspace Structure (simple starter)

project-root/
├── data/          # large raw data (keep out of git; use .gitignore)
├── notebooks/     # exploratory notebooks
├── src/           # reusable functions and pipelines
├── reports/       # exported charts/tables
└── env/           # environment files (requirements.txt, conda.yml)

Keep sample data small and versioned (like the CSVs here); keep production-scale data in object storage or warehouses.
Add a requirements.txt or poetry.lock to freeze dependencies; pin exact versions when collaborating.
Name notebooks with prefixes like 01-eda.ipynb, 02-model.ipynb to show flow; add a short one-line purpose at the top.
Drop a .gitignore entry for data/ (unless you are keeping only tiny samples) and for notebook checkpoints.
Consider a Makefile or simple shell scripts for repeatable tasks (make lint, make test, make notebook).

Practical Checklist

✅ Version control your environment (requirements.txt or poetry.lock).
✅ Enforce dtypes and date parsing on read; log shape and null counts immediately.
✅ Start with asserts and simple profiling (nulls, ranges); fail fast beats silent corruption.
✅ Prefer chains over scattered temporary variables for clarity; factor reusable steps into functions.
✅ Cache interim results to disk (parquet) when they are reused; keep filenames stage-aware (e.g., orders_clean.parquet).
✅ Document assumptions in Markdown cells next to the code; future you will thank present you.
✅ Before modeling, have a crisp question and a success metric; code follows the question, not the other way around.

What’s Next (Preview of Part 3)

Handling missing values and outliers systematically.
Encoding categoricals and first feature engineering patterns.
Practical pandas pipelines for cleaning messy, real-world data.

Part 1 — What Is Data Science? A Complete Beginner-Friendly Overview

Muhammad Arslan — Wed, 03 Dec 2025 09:19:44 GMT

Data science combines statistical thinking, programming, and domain expertise to turn raw data into actionable decisions. If you've ever wondered how Netflix recommends movies, how banks detect fraud, or how companies predict which customers might leave—you're thinking about data science.

This post sets the foundation for the entire series: you'll learn what data science is (and importantly, what it is not), how different roles in the field differ, and where data science delivers real business value. We'll also walk through a hands-on example you can run yourself in a notebook or Python REPL to see data science in action.

Quick TL;DR

Data science is a process: define a question, gather data, clean it, explore, model, deploy, monitor.
It is more than machine learning; business framing, data quality, and iteration drive success.
Roles differ: data scientists focus on questions and models; analysts on reporting; ML engineers on productionizing models.
The best projects start with a clear decision to improve and a metric to move.

A Very Short History

Understanding where data science came from helps explain why the field looks the way it does today. This isn't just academic—knowing the evolution helps you understand why certain tools and practices exist.

1960s–1990s: The Foundation
Statistics matured as a discipline, and database systems (like SQL) became the standard way to store and query structured data. Most analysis happened in Excel or specialized statistical software. Data was relatively small and structured.

2000s: The Big Data Era
Companies started generating massive amounts of data. Technologies like Hadoop enabled distributed storage and processing. Python and R gained traction as powerful, free tools for data analysis. The term "data science" began to emerge.

2010s: The Machine Learning Boom
Cloud computing made powerful infrastructure accessible. GPUs accelerated training of neural networks. Open-source ML libraries (scikit-learn, TensorFlow, PyTorch) democratized machine learning. "Data scientist" became one of the hottest job titles.

2020s: Production and Scale
The focus shifted from building models to deploying and maintaining them reliably (MLOps). Data quality tooling became essential as organizations realized that most ML failures stem from data issues. Large language models (LLMs) opened new possibilities and shifted attention to new interfaces and applications.

What This Means for You: The field is still evolving rapidly. The tools and techniques you learn today will change, but the fundamental principles—framing problems, working with data, and delivering value—remain constant.

What Data Science Really Involves

Contrary to popular belief, data science isn't just about building machine learning models. In fact, modeling often represents less than 20% of a data scientist's time. Here's what the full process actually looks like:

1) Business Framing
Before writing a single line of code, you need to understand the business problem. What decision needs to be improved? For example: "Reduce customer churn by 5% in the next quarter" is a clear, measurable goal. Without this clarity, you risk building elegant solutions to the wrong problem.

2) Data Sourcing
Identify where your data lives: database tables, APIs, CSV files, logs, or third-party sources. Understand who owns the data, how often it updates, and what access you need. This step often involves working closely with data engineers and product teams.

3) Data Quality and Cleaning
Real-world data is messy. You'll spend significant time handling missing values, detecting outliers, aligning units (e.g., converting currencies or timezones), removing duplicates, and enforcing data schemas. This step is critical—garbage in, garbage out.

4) Exploration
Explore your data to understand distributions, identify segments, spot anomalies, and check for data leakage (when future information accidentally leaks into your training data). Visualization is your friend here: histograms, scatter plots, and correlation matrices reveal patterns that numbers alone can't.

5) Modeling (Optional)
Not every data science problem requires a complex model. Start with a simple baseline (like the average, a linear regression, or a rule-based system). Only add complexity if it meaningfully beats your baseline. Many successful projects never use machine learning at all.

6) Delivery
Your work needs to reach decision-makers. This could mean building dashboards, writing reports, creating batch jobs that run automatically, or deploying APIs that serve predictions in real-time. The format depends on who needs the insights and how they'll use them.

7) Monitoring and Iteration
Models degrade over time as the world changes (concept drift). Track data freshness, monitor prediction quality, and measure business impact. Be ready to retrain models, roll back changes, or adjust thresholds as needed.

Roles: Who Does What?

The data science field has several specialized roles. While there's overlap, understanding these distinctions helps clarify career paths and team structures.

Role	Primary Goal	Typical Outputs	Key Skills
Data Scientist	Improve a decision with data and models	Experiments, models, analyses, feature definitions	Statistics, ML, Python/R, business acumen
Data Analyst	Deliver clarity and direction via data	Dashboards, reports, deep dives, SQL queries	SQL, visualization, Excel, domain expertise
ML Engineer	Ship and run models reliably in production	APIs, batch jobs, model serving, monitoring	Software engineering, MLOps, cloud platforms
Data Engineer	Move and organize data for reliability and scale	Pipelines, tables, data contracts, tooling	ETL, databases, distributed systems, Python/Scala

Important Note: In smaller companies or startups, one person might wear multiple hats. A "data scientist" might also build dashboards (analyst work) and deploy models (ML engineer work). In larger organizations, roles are more specialized.

Career Path Insight: Many data scientists start as analysts, learning SQL and business context before moving into modeling. ML engineers often come from software engineering backgrounds and add data science skills. There's no single "right" path—choose based on your interests and strengths.

Real-World Examples (with mini workflows)

1) Reduce Churn in a SaaS App

The Problem: A SaaS company notices customers canceling subscriptions. They want to proactively identify at-risk users and intervene before they churn.

Decision: Keep paying users from canceling by targeting the top-risk 10% with personalized outreach (discounts, feature demos, or support calls).

Data Sources: User events (logins, feature usage), account tenure, payment history, and support ticket volume. These signals help identify patterns that precede churn.

Approach:

Create features that capture user engagement: sessions per week, number of features used, account age, and support interactions
Fit a logistic regression model (a simple, interpretable baseline) to predict churn probability
Rank all users by their predicted churn risk
Focus outreach efforts on the top 10% highest-risk users

Metrics:

Recall@10%: How many of the users who actually churned were in our top 10% risk bucket? (Higher is better)
Net revenue saved: Revenue preserved from prevented churns minus the cost of outreach efforts

Code Example:

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
import pandas as pd

# Load the data
df = pd.read_csv("saas_users.csv")

# Select features (X) and target (y)
# Features: engagement signals that might predict churn
X = df[["sessions_last_7d", "features_used", "tenure_days", "support_tickets"]]
y = df["churned"]  # 1 if churned, 0 if active

# Split data: 80% for training, 20% for testing
# stratify=y ensures both sets have similar churn rates
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Train a logistic regression model
# This learns patterns like: "users with <2 sessions/week are more likely to churn"
model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)

# Get churn probabilities for test users
proba = model.predict_proba(X_test)[:, 1]  # Probability of churning

# Evaluate: AUC measures how well the model ranks risky users
# AUC = 1.0 means perfect ranking, 0.5 means random guessing
print("AUC:", roc_auc_score(y_test, proba))

Why This Works: Users who log in less frequently, use fewer features, or have more support tickets often show early warning signs of disengagement. The model learns these patterns and assigns higher churn risk scores to users matching these patterns.

2) Forecast Weekly Demand for Retail Stores

The Problem: A retail chain needs to stock the right amount of inventory at each store. Too little means stock-outs and lost sales; too much means wasted capital and spoilage.

Decision: Stock stores optimally to minimize stock-outs on fast-moving items while avoiding overbuying slow movers.

Data Sources: Daily sales history, calendar events (holidays, weekends), promotion schedules, and price changes. Historical patterns reveal seasonality and trends.

Approach:

Start with simple but effective baselines: moving averages or exponential smoothing
These methods capture trends and seasonality without complex models
Generate weekly forecasts and compare accuracy against a naive baseline (e.g., "next week = last week")
Only add complexity (like machine learning) if it significantly beats these baselines

Metrics:

MAPE (Mean Absolute Percentage Error): Average forecast error as a percentage (lower is better)
Stock-out rate: Percentage of time items are unavailable when customers want them (lower is better)

Code Example:

import pandas as pd
from statsmodels.tsa.holtwinters import ExponentialSmoothing

# Load daily sales and aggregate to weekly totals
series = (
    pd.read_csv("store_sales.csv", parse_dates=["date"])
      .set_index("date")  # Make date the index for time series operations
      .resample("W")["units_sold"]  # Resample daily data to weekly
      .sum()  # Sum units sold per week
)

# Split: use all but last 4 weeks for training, last 4 for testing
train, test = series[:-4], series[-4:]

# Exponential Smoothing captures:
# - Trend: Is demand growing or declining?
# - Seasonality: Are there weekly/monthly patterns? (52 weeks = yearly seasonality)
model = ExponentialSmoothing(
    train, 
    trend="add",  # Additive trend (demand increases/decreases linearly)
    seasonal="add",  # Additive seasonality (holiday spikes add to baseline)
    seasonal_periods=52  # Yearly patterns (52 weeks)
)
fit = model.fit()
forecast = fit.forecast(len(test))  # Predict next 4 weeks

# Compare forecast to actual test values to measure accuracy

Why This Works: Retail demand often follows predictable patterns: higher sales on weekends, seasonal spikes (holidays, summer), and gradual trends. Exponential smoothing automatically learns these patterns from historical data and projects them forward.

3) Detect Payment Fraud

The Problem: A payment processor needs to identify fraudulent transactions in real-time. Manual review of every transaction is impossible, but missing fraud costs money and customer trust.

Decision: Flag the riskiest transactions for human review without overwhelming the fraud team with false alarms.

Data Sources: Transaction amount, merchant type, device fingerprint, location (GPS/IP), historical user behavior patterns, and chargeback labels (when available for training).

Approach:

Build features that capture suspicious patterns:
- Amount z-score per merchant: Is this transaction unusually large for this merchant?
- Velocity counts: How many transactions did this user make in the last 24 hours? (Fraudsters often make many rapid transactions)
- Distance from user's typical location: Is the transaction happening far from where the user usually shops?
Use Isolation Forest, an unsupervised learning algorithm that identifies outliers without needing labeled fraud examples
Score all transactions and flag the top 0.5% most anomalous for review

Metrics:

Precision@k: Of the top k transactions we flag, how many are actually fraudulent? (Higher is better—we want to minimize false alarms)
Chargeback dollars prevented: Estimated financial impact of catching fraud before it happens

Code Example:

from sklearn.ensemble import IsolationForest
import pandas as pd

df = pd.read_csv("transactions.csv")

# Select features that indicate suspicious behavior
features = df[["amount", "user_txn_count_24h", "merchant_avg_amount", "distance_km"]]

# Isolation Forest: an unsupervised algorithm that finds outliers
# contamination=0.005 means we expect ~0.5% of transactions to be anomalies
clf = IsolationForest(random_state=0, contamination=0.005)

# Fit the model and score transactions
# Returns -1 for normal, 1 for outlier; we flip the sign for easier interpretation
df["anomaly_score"] = -clf.fit_predict(features)  # Higher score = more suspicious

# Get top 0.5% most suspicious transactions for review
review_queue = df.sort_values("anomaly_score", ascending=False).head(
    int(len(df) * 0.005)
)

Why This Works: Fraudulent transactions often look different from normal ones: they might be unusually large, happen in rapid succession, or occur in locations far from the user's typical behavior. Isolation Forest learns what "normal" looks like and flags anything that deviates significantly.

4) Measure Marketing Lift (Incrementality)

The Problem: A marketing team wants to know if their ad campaign is actually driving new sales, or if it's just reaching people who would have bought anyway. This is called "incrementality"—did the ads cause incremental conversions?

Decision: Prove that ads are driving incremental conversions before scaling ad spend. If ads don't drive incremental sales, the budget should be reallocated.

Data Sources: Randomized test/control group assignments (some users see ads, others don't), click data, conversion events, and revenue. Randomization is crucial—it ensures the groups are comparable.

Approach:

Run a controlled experiment: randomly assign users to "test" (see ads) or "control" (no ads) groups
Compute conversion rates for both groups
Calculate the difference (lift) and test if it's statistically significant
If randomized tests aren't feasible, use geo-level experiments (entire cities as test/control) or propensity-matched cohorts

Metrics:

Absolute lift (percentage points): Test conversion rate minus control conversion rate (e.g., 5.2% - 4.0% = 1.2 pp)
Relative lift (%): Absolute lift divided by control rate (e.g., 1.2% / 4.0% = 30% relative lift)
Cost per incremental conversion: Ad spend divided by number of incremental conversions (lower is better)

Code Example:

import pandas as pd
from statsmodels.stats.weightstats import proportions_ztest

df = pd.read_csv("ad_experiment.csv")

# Aggregate conversions by group
# sum = number of conversions, count = total users
conv = df.groupby("group")["converted"].agg(["sum", "count"])

# Statistical test: Is the difference between groups significant?
# This tests: "Could the observed difference be due to random chance?"
stat, p = proportions_ztest(count=conv["sum"], nobs=conv["count"])

# Calculate lift: difference in conversion rates
test_rate = conv.loc["test", "sum"] / conv.loc["test", "count"]
control_rate = conv.loc["control", "sum"] / conv.loc["control", "count"]
lift = test_rate - control_rate

print("Lift (pp):", lift, "p-value:", p)
# If p < 0.05, the lift is statistically significant

Why This Works: By randomly assigning users to test and control groups, we ensure both groups are similar except for exposure to ads. Any difference in conversion rates can be attributed to the ads themselves, not other factors. This is the gold standard for measuring marketing effectiveness.

The Problem: An e-commerce site wants to increase average order value by suggesting relevant add-on products. "Customers who bought X also bought Y" is a classic recommendation problem.

Decision: Show relevant product recommendations to raise average order value without harming conversion rates (recommendations shouldn't distract or annoy users).

Data Sources: Order line items with order_id, user_id, and product_id. This tells us which products are frequently purchased together.

Approach:

Build an item-to-item co-occurrence matrix: count how often each pair of products appears in the same order
For any given product, rank other products by how frequently they co-occur
When a user views or adds a product to cart, show the top co-occurring products as recommendations
A/B test the recommendation widget to measure impact on revenue and conversion

Metrics:

Click-through rate (CTR): Percentage of users who click on recommendations (higher = more engaging)
Incremental revenue per 1,000 sessions: Additional revenue generated from recommendations (higher = more valuable)
Attach rate: Percentage of orders that include a recommended product (higher = more effective)

Code Example:

import pandas as pd
from itertools import combinations
from collections import Counter

orders = pd.read_csv("order_lines.csv")
pairs = Counter()  # Will count how often each product pair appears together

# For each order, find all pairs of products purchased together
for _, group in orders.groupby("order_id")["product_id"]:
    # Get unique products in this order
    products = sorted(group.unique())
    # Count every pair: (product_A, product_B)
    for a, b in combinations(products, 2):
        pairs[(a, b)] += 1

# Function to get top recommendations for a given product
def top_cooccurring(product_id, top_k=5):
    # Find all pairs involving this product and their co-occurrence counts
    scored = [
        (b if a == product_id else a, c)  # Get the "other" product in the pair
        for (a, b), c in pairs.items()
        if product_id in (a, b)
    ]
    # Sort by frequency (most co-occurring first) and return top k
    return sorted(scored, key=lambda x: x[1], reverse=True)[:top_k]

# Example: Get top 5 products that are often bought with product 12
recommendations = top_cooccurring(12, top_k=5)

Why This Works: Products that are frequently purchased together often complement each other (e.g., phone cases with phones, batteries with toys). By analyzing historical purchase patterns, we can identify these relationships and surface them to users at the right moment, increasing the likelihood of additional purchases.

Tiny Hands-On: From Question to Insight

Let's walk through a complete mini-project that demonstrates the data science process from start to finish. This example is simple enough to run in a few minutes but illustrates the key principles.

The Business Question

Question: "Which product categories convert best from email campaigns?"

Context: A marketing team sends promotional emails featuring different product categories (Electronics, Books, Home, Beauty, etc.). They want to know which categories drive the most purchases so they can allocate their email marketing budget more effectively.

Decision to Improve: Focus email campaigns on high-converting categories to maximize return on marketing spend.

The Dataset

We have email_events.csv with the following columns:

user_id: Unique identifier for each user
category: Product category featured in the email (Electronics, Books, Home, etc.)
clicked: 1 if the user clicked the email, 0 if not
purchased: 1 if the user made a purchase, 0 if not

Step-by-Step Analysis

import pandas as pd

# Step 1: Load the data
df = pd.read_csv("email_events.csv")
print(f"Loaded {len(df)} email events")
print(df.head())

# Step 2: Data quality checks
# Verify we have the columns we expect
assert {"category", "clicked", "purchased"} <= set(df.columns), \
    "Missing required columns!"

# Remove rows with missing critical data
# (In real projects, you'd investigate WHY data is missing)
initial_count = len(df)
df = df.dropna(subset=["category", "clicked", "purchased"])
print(f"Removed {initial_count - len(df)} rows with missing data")

# Step 3: Calculate conversion metrics per category
# Group by category and compute average click rate and purchase rate
summary = (
    df.groupby("category")[["clicked", "purchased"]]
      .mean()  # Mean of 0/1 columns = percentage
      .rename(columns={"clicked": "ctr", "purchased": "conversion"})
      .sort_values("conversion", ascending=False)  # Best converters first
)

print("\nConversion Analysis by Category:")
print(summary)

# Step 4: Interpret the results
print("\nTop converting category:", summary.index[0])
print(f"Conversion rate: {summary.iloc[0]['conversion']:.1%}")

What This Example Demonstrates

This simple analysis illustrates several key data science principles:

Start with a business question: We didn't just explore data randomly—we had a clear decision to make (budget allocation).
Data quality matters: Before calculating anything, we checked for missing values. In real projects, you'd also check for duplicates, outliers, and data type issues.
Simple can be powerful: We didn't need machine learning here. A straightforward calculation (average conversion rate per category) answers the question perfectly.
Interpretability: The results are easy to understand and communicate. "Beauty products convert at 100% from email" is clearer than "Model predicts category score of 0.87."
Actionable insights: The output directly informs the decision—focus email campaigns on high-converting categories.

Next Steps (If This Were a Real Project)

Statistical significance: Are the differences between categories statistically significant, or could they be due to small sample sizes?
Segmentation: Do different user segments (new vs. returning, high-value vs. low-value) respond differently to categories?
A/B testing: Test whether focusing on high-converting categories actually increases overall email campaign ROI.
Time analysis: Do conversion rates vary by day of week, time of day, or season?

This example shows that data science doesn't always require complex models—often, the right question and clean data are enough to drive decisions.

How to Judge Success

Not all data science projects succeed. Here's how to evaluate whether your work is making a real impact:

1. Business Impact
The ultimate test: Did the metric tied to your decision actually move? Examples:

Churn reduction project: Did churn decrease by the target amount (e.g., -2% churn rate)?
Conversion optimization: Did conversion rates increase (e.g., +3% conversion)?
Fraud detection: Did chargeback rates decrease while maintaining low false positive rates?

If the business metric didn't improve, the project failed—regardless of how elegant your model was. Always tie your work back to business outcomes.

2. Reliability
Can stakeholders trust your work?

Data freshness: Is the data up-to-date? Stale data leads to bad decisions.
Monitoring: Are you tracking data quality, model performance, and pipeline health?
Reproducibility: Can someone else (or future you) reproduce your results? Document your code, data sources, and assumptions.

3. Communication
Can stakeholders understand and act on your results?

Technical accuracy matters, but if decision-makers can't understand your findings, they won't act on them.
Use clear visualizations, plain language explanations, and concrete recommendations.
A simple dashboard that drives action beats a complex model that sits unused.

4. Simplicity
Did you stop at the simplest approach that works?

Complexity has costs: harder to maintain, explain, and debug.
Start simple (averages, linear models, rule-based systems) and only add complexity if it meaningfully improves results.
Remember: a 5% improvement from a simple model is often better than a 6% improvement from a complex one, if the simple model is easier to deploy and maintain.

The Golden Rule: If your work doesn't change a decision or improve a business metric, it's not successful—no matter how technically impressive it is.

Key Takeaways

Before moving forward, let's recap what we've covered:

Data science is a process, not just modeling. Most time is spent on framing problems, cleaning data, and delivering insights.
Start with business questions, not data. Every project should begin with a clear decision to improve and a metric to move.
Simple solutions often win. Don't default to complex models—start with baselines and only add complexity if it helps.
Different roles serve different purposes. Understanding these roles helps you navigate the field and plan your career.
Success = business impact. Technical excellence matters, but only if it drives real decisions and outcomes.

What's Next (Preview of Part 2)

Now that you understand what data science is and how it works, it's time to get your hands dirty with the tools of the trade.

In Part 2, you'll learn:

How to set up a Python data science environment (pandas, NumPy, Jupyter)
Idiomatic data-wrangling patterns in pandas that will make you productive fast
How to structure a clean project workspace for fast iteration and collaboration
Best practices for writing readable, maintainable data science code

You'll build on the concepts from this post and start working with real data using industry-standard tools. Ready to dive in?

Data Science Mastery: A Complete Learning Path from Beginner to Advanced

Muhammad Arslan — Wed, 03 Dec 2025 09:17:11 GMT

Welcome to Data Science Mastery, a comprehensive 12-part blog series designed to take you from complete beginner to confident data scientist. Whether you're just starting your journey or looking to fill gaps in your knowledge, this series provides a structured, hands-on approach to mastering data science.

Why This Series?

Data science can feel overwhelming. There are countless tools, techniques, and concepts to learn, and it's not always clear where to start or how everything fits together. This series solves that problem by providing:

A clear learning path: Each part builds on the previous one, taking you from fundamentals to advanced topics
Hands-on examples: Every part includes practical code examples and companion notebooks you can run yourself
Real-world focus: Learn techniques that actually work in industry, not just academic exercises
Beginner-friendly: No prior experience required—we explain everything from the ground up
Comprehensive coverage: From data cleaning to model deployment, we cover the full data science lifecycle

Who Is This For?

Complete beginners who want to learn data science from scratch
Career switchers transitioning into data science
Analysts looking to add machine learning to their toolkit
Developers who want to understand data science workflows
Students seeking practical, industry-relevant skills

Prerequisites: Basic comfort with computers and willingness to learn. No programming or math background required—we'll teach you everything you need.

How to Use This Series

Start with Part 1: Even if you have some experience, Part 1 sets the foundation and establishes the mindset
Work through sequentially: Each part builds on previous concepts
Run the code: Don't just read—execute the examples and notebooks
Practice: Apply what you learn to your own datasets
Take your time: Master each part before moving to the next

Estimated timeline:

Parts 1-4 (Fundamentals): 2-3 weeks
Parts 5-7 (Machine Learning): 3-4 weeks
Parts 8-9 (Production): 2-3 weeks
Parts 10-12 (Advanced): 3-4 weeks
Total: 10-14 weeks for complete mastery

The Complete Series

🎯 Foundation Series (Parts 1-4)

Build the fundamentals you'll use every day as a data scientist.

Part 1: What Is Data Science? A Complete Beginner-Friendly Overview

Time: 1-2 hours

History and evolution of data science
What data science really involves (beyond buzzwords)
Data scientist vs analyst vs ML engineer roles
Real-world industry use cases with code examples
How to judge success in data science projects

Key takeaway: Understand what data science is, why it matters, and how it's used in practice.

Part 2: Python for Data Science: Essential Tools and Idioms

Time: 3-4 hours

Why Python dominates data science
pandas, NumPy, and Jupyter essentials
Clean code patterns for data work
How to structure your data science workspace
Idiomatic pandas patterns (assign, pipe, query)

Key takeaway: Get productive with the core Python stack and build good coding habits from day one.

Part 3: Data Cleaning and Preprocessing: 80% of Real Data Science

Time: 4-5 hours

Handling missing values systematically
Detecting and handling outliers
Encoding categorical variables
Feature engineering fundamentals
Building reusable cleaning pipelines

Key takeaway: Master the data cleaning process that takes up most of your time as a data scientist.

Part 4: Exploratory Data Analysis (EDA) With Real Datasets

Time: 4-5 hours

Understanding distributions and relationships
Data visualization (Matplotlib, Seaborn)
Correlation analysis
Building a reusable EDA checklist
Spotting skew, outliers, and leakage risks

Key takeaway: Learn to explore and understand your data before building models.

🤖 Machine Learning Series (Parts 5-7)

Learn to build, evaluate, and optimize machine learning models.

Part 5: Introduction to Machine Learning With Scikit-Learn

Time: 5-6 hours

Classification vs regression problems
Train/test split and cross-validation
Top 5 beginner-friendly algorithms
Practical project: Predict house prices
Model evaluation metrics (MAE, RMSE, R², accuracy, precision, recall)

Key takeaway: Build your first machine learning models and learn to evaluate them properly.

Part 6: Feature Selection and Model Optimization

Time: 4-5 hours

Feature importance and selection techniques
Regularization (L1/L2) explained
Grid search and random search for hyperparameter tuning
Hyperparameter tuning best practices
Avoiding overfitting

Key takeaway: Optimize your models and understand which features matter most.

Part 7: Deep Learning Basics With TensorFlow/PyTorch

Time: 6-8 hours

When to use deep learning vs traditional ML
Neural networks explained visually
Training your first neural network
Experiment tracking with Weights and Biases
Common pitfalls and best practices

Key takeaway: Understand when and how to use deep learning effectively.

🚀 Production Series (Parts 8-9)

Learn to build end-to-end systems and deploy models.

Part 8: Real-World Project: Build an End-to-End ML Pipeline

Time: 6-8 hours

Data ingestion and validation
Preprocessing pipelines
Model training and evaluation
Saving and versioning models
Folder structure and reproducibility
Complete project walkthrough

Key takeaway: Build a production-ready ML pipeline from scratch.

Part 9: Deploying Machine Learning Models

Time: 5-6 hours

Flask/FastAPI deployment
Dockerizing ML applications
Cloud deployment options (AWS, GCP, Azure)
CI/CD for ML (MLOps introduction)
Model monitoring and maintenance

Key takeaway: Deploy your models to production and keep them running reliably.

🎓 Advanced Series (Parts 10-12)

Master advanced techniques and build your career.

Part 10: Advanced Machine Learning Concepts

Time: 6-8 hours

Ensemble methods (Random Forest, XGBoost, LightGBM)
Unsupervised learning (KMeans, DBSCAN, hierarchical clustering)
Time-series forecasting techniques
Modern NLP with transformers
When to use each technique

Key takeaway: Expand your ML toolkit with advanced, production-ready techniques.

Part 11: Data Engineering for Data Scientists

Time: 5-6 hours

ETL vs ELT architectures
Building data pipelines
Airflow basics for workflow orchestration
Working with SQL data warehouses
Why data engineering skills make you a 10x data scientist

Key takeaway: Understand the data infrastructure that powers data science.

Part 12: How to Build a Data Science Portfolio and Get Hired

Time: 4-5 hours

Must-have projects for your portfolio
Writing impactful case studies
GitHub structure and best practices
Interview prep: ML, statistics, system design for data science
How to stand out in the 2025+ job market

Key takeaway: Build a portfolio that gets you hired and ace your interviews.

📚 Additional Real-World Use Case Series (Parts 13-17)

Once you've mastered the fundamentals, dive deep into specific industry use cases with these advanced projects.

Part 13: Churn Prediction for SaaS

Define churn, label windows, and target metrics
Feature engineering from product usage and support signals
Baseline logistic regression, calibration, and uplift-focused evaluation
Playbook for outreach experiments based on risk tiers

Part 14: Demand Forecasting for Retail

Hierarchical time series (store/item) and calendar effects
Baselines (naive, moving average, ETS) vs. gradient boosting
Promotion/price feature engineering and holidays
Metrics: MAPE/MASE and stock-out/carrying cost tie-ins

Part 15: Fraud Detection for Payments

Feature building: velocity, device/geo, merchant norms
Imbalanced learning (class weights, focal loss, anomaly scores)
Precision@k and dollar-weighted metrics for review queues
Latency considerations for online scoring

Part 16: Recommendation Systems for E-commerce

Item-item co-occurrence and matrix factorization primers
Cold start tactics (content-based, popularity priors)
A/B testing recommendations and guardrails for diversity
Offline/online metrics: CTR, conversion lift, coverage, novelty

Part 17: Forecasting and Anomaly Detection for Operations

Rolling forecasts for capacity/SLAs; detecting anomalies in KPIs
Seasonality/trend decomposition; robust thresholds
Alert design: precision/recall tradeoffs for incidents
Runbooks for investigation and feedback loops to the model

🎁 Bonus Series (Optional Add-Ons)

ML Ops in Practice - A 10-Part Series

Deep dive into production ML: model versioning, monitoring, A/B testing, and more.

Deep Learning From Scratch

Build neural networks from the ground up, understanding every component.

Domain-Specific Series

Data Science for Finance
Data Science for E-commerce
Data Science for Healthcare

AI Agentic Systems for Data Science Workflows

Cutting-edge techniques for automating data science workflows with AI agents.

Learning Path Recommendations

🟢 Beginner Path (Complete Newcomer)

Start with Parts 1-4 (Foundation)
Complete Parts 5-6 (ML Basics)
Build a portfolio project using Parts 1-6
Continue with Parts 7-9 as you're ready
Timeline: 3-4 months

🟡 Intermediate Path (Some Experience)

Review Parts 1-2 (ensure fundamentals are solid)
Focus on Parts 3-6 (Data work + ML)
Deep dive into Parts 7-9 (Advanced ML + Production)
Complete Parts 10-12 (Advanced topics + Career)
Timeline: 2-3 months

🔴 Advanced Path (Experienced Practitioner)

Skim Parts 1-4 (refresh fundamentals)
Focus on Parts 8-9 (Production systems)
Master Parts 10-12 (Advanced techniques)
Complete use case series (Parts 13-17)
Timeline: 1-2 months

What Makes This Series Different?

✅ Practical Focus

Every concept is demonstrated with real code and real datasets. You'll build projects, not just read theory.

✅ Industry-Relevant

Learn techniques actually used in production, not just academic exercises. We focus on what works in the real world.

✅ Beginner-Friendly

No assumptions about prior knowledge. We explain everything from first principles, with clear examples.

✅ Comprehensive

Covers the full data science lifecycle: from raw data to deployed models to career advancement.

✅ Reproducible

All code is provided, all datasets are included, and everything is version-controlled. You can run everything yourself.

✅ Community-Driven

Based on real questions from data scientists at all levels. We address the problems you actually face.

Getting Started

Set up your environment (Part 2 covers this in detail):

python3 -m venv .venv
source .venv/bin/activate
pip install pandas numpy jupyterlab matplotlib seaborn scikit-learn

Start with Part 1: Read the blog post and run the examples
Work through sequentially: Each part builds on the previous
Join the community: Share your progress, ask questions, help others

Resources and Support

All code is on GitHub: Clone the repo and run everything locally
Companion notebooks: Each part includes Jupyter notebooks you can execute
Sample datasets: Real-world datasets included for practice
Makefiles: Quick setup scripts for each part

Frequently Asked Questions

Q: Do I need a math background?
A: No! We explain concepts intuitively. Math helps, but we focus on practical understanding.

Q: How long does the full series take?
A: 10-14 weeks if you follow the recommended timeline. But go at your own pace!

Q: Can I skip parts?
A: We recommend going sequentially, but if you have experience, you can skip ahead. Just be sure you understand the prerequisites.

Q: What if I get stuck?
A: Each part includes troubleshooting tips. The code is well-commented and the notebooks are self-contained.

Q: Is this enough to get a job?
A: Combined with practice projects and a portfolio (Part 12 covers this), yes! Many students have successfully transitioned into data science roles.

Q: Do I need expensive software?
A: No! Everything uses free, open-source tools. You can run everything on your laptop.

Ready to Start?

👉 Begin with Part 1: What Is Data Science?

Take your first step into data science. No prior experience needed—just curiosity and a willingness to learn.

Stay Updated

Bookmark this page: Your roadmap through the entire series
Follow along: Work through parts sequentially
Practice: Apply concepts to your own projects
Share: Help others on their data science journey

Remember: Data science is a journey, not a destination. Every expert was once a beginner. Start with Part 1, take it one step at a time, and before you know it, you'll be building production ML systems.

Let's begin! 🚀

Last updated: 2025. This series is continuously improved based on feedback from the data science community.

Introducing CIDR: A Beautiful and Interactive CIDR Calculator

Muhammad Arslan — Tue, 02 Dec 2025 09:03:32 GMT

What is CIDR?

CIDR (Classless Inter-Domain Routing) is a method for allocating IP addresses and routing Internet Protocol packets. It's a fundamental concept in networking that helps in efficient IP address allocation and routing. Today, I'm excited to introduce a modern, interactive CIDR calculator that makes working with IP addresses and subnet calculations a breeze. And it is open source github

Features

The CIDR Calculator comes with a beautiful, modern interface and offers several powerful features:

Interactive IP Address Input

Easy-to-use octet-based input system
Real-time validation of IP addresses
Visual binary representation of IP addresses

2. Comprehensive CIDR Information

Network address calculation
Broadcast address
Subnet mask
First and last usable host addresses
Total number of available hosts

Advanced Information

Wildcard mask
IP class identification
Host address range
Reverse DNS (PTR) records
Binary and hexadecimal representations

Practical Examples

Let's look at some practical examples of how to use the CIDR Calculator:

Example 1: Basic Network Configuration

Let's say you want to set up a network with the IP address 192.168.1.0/24:textApply to package.jsonIP Address: 192.168.1.0Prefix Length: 24The calculator will show you:

Network Address: 192.168.1.0
Broadcast Address: 192.168.1.255
Subnet Mask: 255.255.255.0
First Usable IP: 192.168.1.1
Last Usable IP: 192.168.1.254
Total Hosts: 254

Example 2: Smaller Subnet

For a smaller network, you might use 10.0.0.0/28:textApply to package.jsonIP Address: 10.0.0.0Prefix Length: 28This will give you:

Network Address: 10.0.0.0
Broadcast Address: 10.0.0.15
Subnet Mask: 255.255.255.240
First Usable IP: 10.0.0.1
Last Usable IP: 10.0.0.14
Total Hosts: 14

Example 3: Large Network

For a larger network, you might use 172.16.0.0/16:textApply to package.jsonIP Address: 172.16.0.0Prefix Length: 16This will show:

Network Address: 172.16.0.0
Broadcast Address: 172.16.255.255
Subnet Mask: 255.255.0.0
First Usable IP: 172.16.0.1
Last Usable IP: 172.16.255.254
Total Hosts: 65,534

Key Benefits

User-Friendly Interface

Clean, modern design
Real-time calculations
Visual binary representation
Copy-to-clipboard functionality

2. Shareable Results

Generate shareable links
Copy CIDR notation
Export results

3. Advanced Features

IP class detection
Binary and hex conversions
PTR record generation
Wildcard mask calculation

Getting Started

You can start using the CIDR Calculator right away by visiting https://github.com/muhammadarslan/cidr. The interface is intuitive and requires no installation. Simply:

Enter your IP address
Select your desired prefix length
View the comprehensive results

Conclusion

The CIDR Calculator is a powerful tool for network administrators, developers, and anyone working with IP addressing. Its modern interface and comprehensive feature set make it an essential tool for network planning and troubleshooting.Try it out today and simplify your CIDR calculations!

https://cidr.jhear.com

gitcli: AI-First Git Productivity in Your Terminal

Muhammad Arslan — Mon, 01 Dec 2025 09:03:20 GMT

gitcli is a TypeScript-powered, Commander.js CLI that layers AI helpers, automation, and plugins on top of git. It’s published on npm as gitcli, so you can install and use it globally:

npm install -g gitcli
gitcli --help

Why gitcli

Faster commits: AI-generated messages (gitcli commit --ai --style conventional|casual|emoji) plus local notes with --note.
Safer changes: AI reviews (gitcli review), conflict suggestions (gitcli resolve --ai), secret scanning (gitcli scan), and protected-branch warnings.
Better flow: predictive next-command hints, history analytics, fish-style inline suggestions, and fuzzy correction for typos.
Cleaner repos: dirty file detector and impact hints (gitcli impact), branch hygiene (gitcli clean), and tamper detection.
Extensible: plugin loader (~/.gitcli/plugins), install via gitcli plugin install .

AI Features

Commit helper reads staged diff and drafts messages.
Code review for staged changes with concise suggestions.
Merge conflict resolver suggests merged content for conflicted files.
Branch name generator from a task description.
PR summary generator turns recent commits into PR title/description and can notify Slack/Discord when configured.

Developer Experience

TUI dashboard (gitcli ui) shows status, recent commits, and branch graph.
Command history analytics (gitcli history stats|list|clear) plus next-command predictions.
Auto-correct for mistyped commands and inline history hints on startup.

Automation & Hygiene

Smart pre-commit tasks runner (gitcli hooks) for Prettier, ESLint, tests.
Smart dirty file detector inside gitcli impact with change area hints and cross-repo awareness via config.
Branch cleanup (gitcli clean) and tamper detection warnings.

Security & Release

Secret scanner on staged files (gitcli scan).
Encrypted stash (gitcli stash --secure [-p password]).
Changelog + semantic bump advice (gitcli release create), remote git execution, and stash/remote helpers.

Collaboration & Cloud

Pair mode over WebSocket (gitcli pair start|stop).
PR notifications (Slack/Discord) for summaries.
Cloud sync (gitcli sync) pushes history/aliases to your configured endpoint.

Install & Link Locally

npm install
npm run build
npm link

Testing

Run Jest tests (ESM + ts-jest):

npm test

Config

Stored at ~/.gitcli/config.json:

{
  "aiProvider": "openai",
  "aiModel": "gpt-3.5-turbo",
  "aiBaseUrl": "https://api.openai.com/v1",
  "tokens": {"openai": "sk-..."},
  "linkedRepos": ["../other-repo"],
  "pluginPolicies": { "allow": ["my-plugin.js"], "deny": [] },
  "notifications": {"slackWebhook": "", "discordWebhook": ""},
  "cloudSync": {"enabled": false, "endpoint": "https://api.example.com/gitcli/sync", "apiKey": "token"}
}

AI setup: set OPENAI_API_KEY (or tokens.openai in config). Optionally set OPENAI_BASE_URL for proxy-compatible endpoints. Fallback behavior logs a stubbed response if the API call fails.

Plugins: optional allow/deny lists via pluginPolicies to skip untrusted plugins.
Sandboxing: plugins run in a VM context without require or fs; only Command/console exposed.

Secret scan: maintain .gitcli-scan-allowlist for known benign matches.

Hooks: gitcli hooks run runs Prettier/ESLint/tests; gitcli hooks install adds a git pre-commit hook to run npm test.
Platform: hook install writes both sh (pre-commit) and PowerShell (pre-commit.ps1) to support macOS/Linux and Windows (Git for Windows defaults to sh).

Architecture Snapshot

ES modules + TypeScript compiled to dist/.
src/commands for Commander commands; src/ai for AI stubs; src/utils for config/history/security/git wrappers; src/plugins/loader.ts auto-loads ~/.gitcli/plugins; src/tui/dashboard.ts powers the Blessed UI.
State lives in ~/.gitcli (config, history, notes, plugins, secure stash).

A Day With gitcli

Start a feature: gitcli branch create "add billing events" --ai
Stage and review: gitcli review
Commit with AI + note: gitcli commit --ai --note "paired with sam"
Check impact and hygiene: gitcli impact && gitcli clean
Summarize for PR and notify: gitcli pr summarize --notify
Quick glance at repo health: gitcli ui

gitcli keeps you in flow—fewer context switches, smarter defaults, and a plugin-ready core. Install from npm as gitcli and bring AI, automation, and team-aware tooling to your terminal.

Benefits at a Glance

Ship faster: AI commit messages, reviews, and branch naming reduce keystrokes and context-switches.
Safer releases: secret scanning, tamper warnings, protected-branch reminders, and dirty file detection prevent risky pushes.
Cleaner repos: automated branch cleanup and impact hints keep history tidy.
Team visibility: PR summaries with notifications, pair mode, and history analytics improve collaboration.
Extensible foundation: plugin loader and config-driven cloud sync mean the CLI grows with your workflow.

Command Reference (Key Highlights)

gitcli commit [--ai --style conventional|casual|emoji --note --all --message ]
Stage optionally, warn on protected branches, generate AI message if requested, and save local notes per commit.
gitcli review
Runs AI review on staged diff and prints concise, actionable suggestions.
gitcli resolve --ai
For each conflicted file, generates suggested merged content.
gitcli branch create [--ai --name ]
Creates a branch; AI mode turns description into kebab-case name.
gitcli pr summarize [--count --notify]
Summarizes recent commits into PR title/description; optional Slack/Discord notify via config webhooks.
gitcli ui
Blessed dashboard with repo status, recent commits, and branch graph; quit with q, Esc, or Ctrl+C.
gitcli history stats|list|clear
Stats show totals, top commands, average session length, and recent entries; list shows recent commands; clear resets history.
gitcli clean [--prune-remote]
Deletes merged local branches (excluding main/master) and optionally prunes remotes.
gitcli impact
Analyzes staged files, highlights likely impact areas, warns on modified key config/test files, and lists linked repos from config.
gitcli plugin install | list
Install ES module plugins into ~/.gitcli/plugins or list installed plugins.
gitcli pair start | stop
Lightweight pair-programming session over WebSocket; logs incoming messages and closes on stop.
gitcli remote
Executes git commands over SSH via ssh git ....
gitcli scan
Secret scan on staged files using heuristic regexes; exits non-zero on findings.
gitcli stash --secure [-p ]
Encrypts staged diff with AES-256 into ~/.gitcli/secure-stash and hard-resets; falls back to git stash when not secure.
gitcli release create [--tag --publish --repo owner/repo --draft]
Generates changelog and manifest; can publish to GitHub using GITHUB_TOKEN (or tokens.github) and origin-derived repo.
gitcli fun streak|art|health
Streak: days with commits; art: ASCII heatmap from recent commits; health: simple repo hygiene score.
gitcli hooks run|install
Run Prettier/ESLint/tests or install a git pre-commit hook to run npm test.
gitcli sync
Posts history/aliases to cloudSync.endpoint with optional Bearer API key.

Install from npm as gitcli, or develop locally with npm install && npm run build && npm link. The CLI is ready to plug into your existing repos and extend via plugins and config.

Lattice: Scale Application over docker

Muhammad Arslan — Tue, 26 Oct 2021 11:54:47 GMT

Lattice is a cloud-native application platform that enables you to run your applications in containers using solutions like Docker.

Lattice includes features like:

Cluster scheduling
HTTP load balancing
Log aggregation
Health management

Lattice does this by packaging a subset of the components found in the Cloud Foundry elastic runtime. The result is an open, single-tenant environment suitable for rapid application development. Applications developed using Lattice should migrate unchanged to full Cloud Foundry deployments.

Install Lattice:

Download and install Virtual box
Download and install Vagrant
Download and install git
git clone https://github.com/cloudfoundry-incubator/lattice.git
cd lattice
vagrant up –provider virtualbox (run this command from lattice folder)
This will take some time, when finished will show the message something like this.

192.168.11.11.xip.io. Note it for later usage.

Deploying a Containerized Spring Cloud app with Lattice

With Lattice set up, you will now install a Spring Cloud sample application into your lattice installation from Docker’s hub location.

Point ltc at your Lattice setup by typing: ltc target
LATTICE_CLI_TIMEOUT=180 ltc create spring-cloud-lattice-sample springcloud/spring-cloud-lattice-sample --memory-mb=0(0 means no memory limits. See http://lattice.cf/docs/ltc/ for more details.)
Scale the app to three instances by typing ltc scale spring-cloud-lattice-sample 3
Visit http://spring-cloud-lattice-sample.192.168.11.11.xip.io?service=spring-cloud-lattice-sample and verify you can see the JSON service record. Refresh the browser multiple times notice how the uri attribute rotates.
Visit http://spring-cloud-lattice-sample.192.168.11.11.xip.io/me and see a pared down record that also rotates the uri.

See http://lattice.cf/docs/ltc/ for more details about the various CLI commands you can run.

Java 8: New Features

Muhammad Arslan — Tue, 26 Oct 2021 11:54:13 GMT

Default Methods or Extension Methods for Interfaces

Java 8 allows us to add non-abstract method implementation to interfaces by utilizing the default keyword.

Here is our first example:

interface Calculator {
    double calculateSqrt(int a);
 
    default double sqrt(int a) {
        return Math.sqrt(a);
    }
}

Besides abstract calculateSqrt method, there is one non-abstract default method with implementation. Concrete class which implements this interface just needs to implement calculateSqrt method.

Here’s the example showing anonymous object implementation.

final Calculator calculator = new Calculator() {
         @Override
	 public double calculateSqrt(int a) {
		return sqrt(a * 100);
         }
};

calculator.sqrt(16);           // 4.0 
calculator.calculateSqrt(16);  // 40.0

Quite allot of code for method implementation. We will see later how lambda expression helps us to implement one method object.

Functional Interfaces

All java developer should have used interfaces at least one of the following: java.lang.Runnable, java.util.Comparator, java.util.concurrent.Callable etc. These interfaces has some common features which is they have exactly one method defined. These interfaces are also called Single Abstract Method interfaces (SAM Interfaces). These interfaces can be used as annonymous inner classes.

public class TemperatorConverter {
  public static void main(String[] args) {
    new Thread(new Runnable() {
      @Override
      public void run() {
        System.out.println("New thread is created in Temperature Converter...");
      }
    }).start();
  }
}

In Java 8 these interfaces are recreated and standardized as Functional Interfaces. There is new annotation @FuncationInterface introduced, used for compile level errors and validate as Functional Interface. Functional interface should have exactly one method, compiler is aware of this annotation and warns you if you want to add another method.

@FuncationalInterface
public interface TemperatureConverter {
       C convertFahrenheitToCelsius(F From);
}

The code is also valid if you remove @FuncationalInterface annotation, but no compile time validation.

TemperatureConverter tempConverter = (from) -> ((from - 32)*5)/9;
Double celciousTemp = tempConverter.convertFahrenheitToCelsius(100.0);
System.out.println(celciousTemp); // 37.77

Static JavaScript code Analysis

Muhammad Arslan — Tue, 26 Oct 2021 11:51:19 GMT

As JavaScript might be the world’s most misunderstood language, but it is one of most popular and used one. Then you need to deal this in more nice way, better structure of code, unit and automated tests etc. So the available tools helps you manage your code in more managed way.

You can’t control what you can’t measure Tom DeMarco

This quote inspired me while i was studying. Now i found this helpful in my career. But this is not easy, you need to collect data, visualise it, do some analysis and take some good decisions.

Plato gives you static JavaScript analysis and reporting for free.

Create an app with angular-fullstack. Refer to post Modern development stacks: Boilerplate

Install it in the folder which code you want to visualise. For example we take angular-fullstack app.

npm install plato

And then execute

node_modules/.bin/plato -r -d server-report -t 'Server' -x .json ./server

As a result you’ll get an amazing report with dozens of different visualizations of your data, generated directly into your specified output folder server-report. Here are some screen shots.

Conclusion

Try this and see if this make sense for you. Developer’s job is not only meet customer’s requirement, but also needs to write the code which is maintainable and extendable.

Plato makes it extremely easy to generate a report, so the real challenge is to interpret the results, identify potential weaknesses and sanitize them accordingly.

You can use this as part of your continuous integration cycle, run this analysis on each commit and push the report somewhere on webserver, so that all developers can see and improve the code.

Build resume with json

Muhammad Arslan — Tue, 26 Oct 2021 11:48:41 GMT

Introduction

If you’re a developer geek like I am, you’ll want to check this out. This is awesome tool for creating resume with json file, cv can be exported as pdf or html. You can also apply different themes to make your resume looks nicer.

Only thing you need to do it create a json file. You can also host json as git repository and have a continuous deployment by just updating json file. Like a developer geek you want to have something that follows your daily work routine.

You can deploy html version in any of the cloud service.

Installation

You can install as npm cli package, by simply executing this command.

sudo npm install -g resume-cli

Create

You can create sample json resume file.

resume init

You can also and build json file online, by clicking following link.

http://registry.jsonresume.org/

Export

You can export pdf resume.

resume export resume.pdf

You can also export it as html.

resume export resume.html

OCPJP: Preparation and Exeperience

Muhammad Arslan — Tue, 26 Oct 2021 11:46:47 GMT

Keep in Mind before starting preparation

The certification exam is not easy, preparation is time consuming even you experienced java developer.
You will feel that you really improved your programming and analytical skills.
You will improve your thinking process, how you percieve the problem and how you solve it.
You will stand different from the crowd.
Certification is your proof that you know java well.

Preparation and Resources

Read K&S book well, understand each line and concept. Don’t care if it takes more time to understand.

I read this book 3-4 times thoroughly. On each time i learned something new. This book has quite allot of information.
The exam contains questions from all the sections, read outline of each section and prepare accordingly.
If you feel you are weak in any of the section, spend more time on it, till you feel comfortable.
Do not go to any other resource until you finish reading this book and exercies, that will make you confuse. This book alreay has all the concepts required in this exam.
Make some notes of the points that you want to remember, in my case i put points on stickys and i read during my free time in office and on the way coming/back from office.
After reading the book, do allot of mock exams.

The resources which i tried for mock questions.

Both resources are good and have quite allot of mock questions.
Do not try mock exams until you don’t read the book thoroughly, that will make yo confuse.

If you get a black belt, then you are good to go for exam, and that is even worth same like OCPJP.

Revise book in last week before exam, do not spend too much time, few hours on each sections. In my case i did it in early mornings, more energy and fresh mind.

Note: If you are just preparing for exam, do not go to any other resource then book, because this book has all the information needed for exam.

About exam

Do not read anything the morning when you have the exam. Fresh your mind, drink tea or coffee before exam and go with full of confidance.
Forget that if the exam is easy, it is difficult, in my case some of the questions were very easy that i answered in just few seconds. But some of the questions took allot of time to answer.
Skip the questions if you don’t comfortable with answer, answer the easiest questions first. After finishing easiest questions comback to the skipped one’s, and answer accordingly.