Word Frequency solution and explanation
Google
3 more videos on this problem
Leetcode 192 - Word Frequency | Solving leetcode problems using bashDaigen B. River · 3:01 · 836 views
Word Frequency (LeetCode) Problem | Bash scriptingSridhar Media · 9:57 · 196 views
Word Frequency Counter | Python | LeetCode 192MidnightCodeLab · 3:17 · 49 views
No Sound Leetcode 192: Word FrequencyExtremeProgrammer · 3:24 · 35 views
The problem
Write a bash script to calculate the frequency of each word in a text file words.txt.
For simplicity sake, you may assume:
words.txtcontains only lowercase characters and space' 'characters.- Each word must consist of lowercase characters only.
- Words are separated by one or more whitespace characters.
Example:
Assume that words.txt has the following content:
the day is sunny the the the sunny is is
Your script should output the following, sorted by descending frequency:
the 4 is 3 sunny 2 day 1
Note:
- Don't worry about handling ties, it is guaranteed that each word's frequency count is unique.
- Could you write it in one-line using Unix pipes?
Approach overview
Problem Overview: You receive a text file containing words separated by spaces. The task is to count how many times each word appears and print the result sorted by frequency in descending order. The challenge focuses on efficient text processing using shell utilities or frequency counting techniques.
Approach 1: Using Command-line Utilities (O(n log n) time, O(n) space)
This approach uses standard Unix text-processing tools. First normalize the input so each word appears on its own line using tr or similar utilities. Then sort the words with sort, count duplicates using uniq -c, and finally sort again by frequency in descending order. The key insight is that uniq only counts adjacent duplicates, so sorting groups identical words together. This pipeline-based solution is concise and idiomatic for shell scripting tasks.
Approach 2: Using grep, sort, and uniq (O(n log n) time, O(n) space)
Another shell-based solution relies on grep -o to extract individual words from the file, printing each match on a new line. Once extracted, the pipeline continues with sort to group identical words and uniq -c to compute counts. A final descending sort arranges results by frequency. This approach works well when the input format may contain punctuation or irregular spacing, because grep can precisely control the word pattern being extracted.
Approach 3: Recursive Approach with Memoization (O(n) time, O(k) space)
If implemented in languages like Python or Java, the problem becomes a classic frequency counting task. Iterate through the list of words and store counts in a hash map. A recursive helper can process one word at a time while memoizing previously seen counts in the map. Each insertion or lookup in the hash table runs in average O(1) time. After counting, convert the map entries into a list and sort them by frequency before printing.
Approach 4: Iterative Dynamic Programming (O(n) time counting + O(k log k) sorting, O(k) space)
An iterative approach processes the text sequentially and updates a frequency table for each encountered word. The “DP” aspect is simply building results from previous counts: each step updates count[word] = count[word] + 1. The main data structure is a hash map keyed by the word string. Once all words are processed, sort the unique entries by frequency. This approach is common in general string processing problems and works well outside shell environments.
Recommended for interviews: Interviewers usually expect the hash map frequency-count solution because it demonstrates understanding of constant-time lookups and sorting results by value. Showing the shell pipeline solution proves strong command-line skills, but the hashmap approach highlights core algorithmic thinking.
Approach 1: Using Command-line Utilities
This approach uses a combination of Unix command-line utilities to achieve the desired result. It leverages the pipeline mechanism where the output of one command becomes the input to the next logically and efficiently.
#!/bin/bash
awk '{ for(i=1;i<=NF;i++) freq[$i]++ } END { for(word in freq) print word, freq[word] }' words.txt | sort -k2,2nrThis solution uses `awk` to scan each word in the file and maintain a frequency count in an associative array, `freq`. The `END` block prints each word and its count. This output is then piped into the `sort` command, which sorts the results numerically in descending order by the second column (the frequency values).
The time complexity is O(n log n), where `n` is the number of unique words due to sorting. The space complexity is O(k), where `k` is the number of unique words, since it stores the frequency of each word.
Approach 2: Using `grep`, `sort`, and `uniq`
This approach relies on manually splitting input into words and using sorting to order them and then using `uniq` to count the frequencies.
#!/bin/bash
tr -s ' ' '\n' < words.txt | sort | uniq -c | sort -nr | awk '{print $2, $1}'This solution uses `tr` to replace spaces with newlines, effectively breaking words into individual lines. The output is piped into `sort` to arrange the words alphabetically. The `uniq -c` command counts consecutive identical words. Another `sort` is then used to order the results by frequency in descending order. Finally, `awk` is used again to print the final output format with the word before the frequency count.
The time complexity is O(n log n) primarily due to the sort operations. The space complexity is O(k), where `k` is the number of unique words because of the storage for word frequency.
Approach 3: Recursive Approach with Memoization
This approach involves breaking the problem into smaller subproblems that can be solved recursively. By using memoization, we store the results of subproblems to avoid redundant calculations, thereby optimizing the solution.
def fib(n, memo={}):
if n in memo:
return memo[n]
if n <= 1:
return n
memo[n] = fib(n - 1, memo) + fib(n - 2, memo)
return memo[n]
print(fib(10))import java.util.Arrays;
public class Fibonacci {
static int[] dp = new int[1000];
public static int fib(int n) {
if (n <= 1) return n;
if (dp[n] != -1) return dp[n];
return dp[n] = fib(n - 1) + fib(n - 2);
}
public static void main(String[] args) {
Arrays.fill(dp, -1);
int n = 10;
System.out.println(fib(n));
}
}#include <iostream>
#include <vector>
using namespace std;
vector<int> dp(1000, -1);
int fib(int n) {
if(n <= 1)
return n;
if(dp[n] != -1)
return dp[n];
return dp[n] = fib(n - 1) + fib(n - 2);
}
int main() {
int n = 10;
cout << fib(n) << endl;
return 0;
}const fib = (n, memo = {}) => {
if (n in memo) return memo[n];
if (n <= 1) return n;
return memo[n] = fib(n - 1, memo) + fib(n - 2, memo);
};
console.log(fib(10));#include <stdio.h>
#define MAX 1000
int dp[MAX];
int fib(int n) {
if (n <= 1)
return n;
if (dp[n] != -1)
return dp[n];
return dp[n] = fib(n-1) + fib(n-2);
}
int main() {
for (int i = 0; i < MAX; i++) dp[i] = -1; // Initialize memoization array
int n = 10;
printf("%d\n", fib(n));
return 0;
}using System;
using System.Collections.Generic;
class Fibonacci {
static Dictionary<int, int> memo = new Dictionary<int, int>();
static int Fib(int n) {
if (n <= 1) return n;
if (memo.ContainsKey(n)) return memo[n];
memo[n] = Fib(n - 1) + Fib(n - 2);
return memo[n];
}
static void Main() {
Console.WriteLine(Fib(10));
}
}This Python code uses a dictionary for memoization, passing it along recursively to store and access previously computed Fibonacci values efficiently.
Time Complexity: O(n) due to memorized results.
Space Complexity: O(n) for the dictionary used in memoization.
Approach 4: Iterative Dynamic Programming
The iterative dynamic programming approach calculates the Fibonacci numbers from the bottom up, using loops to build the result from the base cases. It avoids recursion and efficiently computes the result using a constant amount of space (for the simple implementation).
def fib(n):
if n <= 1:
return n
a, b = 0, 1
for _ in range(2, n + 1):
a, b = b, a + b
return b
print(fib(10))class Fibonacci {
public static int fib(int n) {
if (n <= 1) return n;
int a = 0, b = 1, c = 0;
for (int i = 2; i <= n; i++) {
c = a + b;
a = b;
b = c;
}
return c;
}
public static void main(String[] args) {
int n = 10;
System.out.println(fib(n));
}
}#include <iostream>
using namespace std;
int fib(int n) {
if(n <= 1)
return n;
int a = 0, b = 1, c;
for(int i = 2; i <= n; i++) {
c = a + b;
a = b;
b = c;
}
return b;
}
int main() {
int n = 10;
cout << fib(n) << endl;
return 0;
}const fib = (n) => {
if (n <= 1) return n;
let a = 0, b = 1, c;
for (let i = 2; i <= n; i++) {
c = a + b;
a = b;
b = c;
}
return b;
};
console.log(fib(10));#include <stdio.h>
int fib(int n) {
if (n <= 1)
return n;
int a = 0, b = 1, c;
for (int i = 2; i <= n; i++) {
c = a + b;
a = b;
b = c;
}
return b;
}
int main() {
int n = 10;
printf("%d\n", fib(n));
return 0;
}using System;
class Fibonacci {
public static int Fib(int n) {
if (n <= 1) return n;
int a = 0, b = 1, c = 0;
for (int i = 2; i <= n; i++) {
c = a + b;
a = b;
b = c;
}
return c;
}
static void Main() {
Console.WriteLine(Fib(10));
}
}This Python implementation leverages an iterative loop to compute Fibonacci numbers, maintaining only the last two values, thus conserving space.
Time Complexity: O(n)
Space Complexity: O(1)
Complexity comparison
| Approach | Complexity |
|---|---|
| Using Command-line Utilities | The time complexity is O(n log n), where `n` is the number of unique words due to sorting. The space complexity is O(k), where `k` is the number of unique words, since it stores the frequency of each word. |
| Using `grep`, `sort`, and `uniq` | The time complexity is O(n log n) primarily due to the sort operations. The space complexity is O(k), where `k` is the number of unique words because of the storage for word frequency. |
| Recursive Approach with Memoization | Time Complexity: O(n) due to memorized results. |
| Iterative Dynamic Programming | Time Complexity: O(n) |
Complexity analysis
| Approach | Time | Space | When to Use |
|---|---|---|---|
| Command-line Utilities Pipeline | O(n log n) | O(n) | Best for shell scripting and quick text processing using Unix tools |
| grep + sort + uniq | O(n log n) | O(n) | When word extraction requires pattern matching or flexible parsing |
| Recursive Hash Map Counting | O(n) + O(k log k) | O(k) | General programming solution using hash tables |
| Iterative Frequency Table (DP-style) | O(n) + O(k log k) | O(k) | Preferred in interviews for efficient counting and clear logic |
Frequently asked questions
Is Word Frequency easy or hard?
The problem is rated Medium mainly because of shell command composition and output formatting. The underlying concept—counting frequencies with a hash map—is straightforward once you recognize the pattern.
Word Frequency Python/Java solution
In Python or Java, split the input text into words and store counts in a dictionary or HashMap. Increment the count for each word during iteration, then sort the entries by frequency before printing the results.
How to solve Word Frequency in O(n)?
The counting phase can be done in O(n) by iterating through all words and storing their counts in a hash table. Each insertion or update takes constant average time. If the output does not require sorting, the entire operation stays O(n); otherwise sorting adds O(k log k).
What is the best approach for Word Frequency?
The most practical approach is counting words using a hash map and then sorting the results by frequency. Each word lookup and update runs in O(1) average time, making the counting step O(n). After counting, sorting the unique words takes O(k log k), where k is the number of distinct words.
Is Word Frequency asked at Google/Amazon/Meta?
Frequency counting problems are common interview patterns across companies like Amazon, Google, and Meta. Variations appear in text processing, log analysis, and string manipulation questions that require hash maps and sorting by frequency.
What data structure is used in Word Frequency?
A hash table (hash map or dictionary) is the primary data structure. It maps each word to the number of times it appears, enabling constant-time updates and lookups while processing the text.
What is the time complexity of Word Frequency?
Shell-based solutions typically run in O(n log n) time because they rely on sorting utilities like sort before applying uniq. Hash map counting solutions run in O(n) for counting plus O(k log k) for sorting unique words by frequency.