Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/build_test_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,14 @@ jobs:
git

- name: Configure and Build
working-directory: heiFIP
run: |
cd heiFIP/
cmake -B build -S . \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_TOOLCHAIN_FILE="$VCPKG_INSTALLATION_ROOT/scripts/buildsystems/vcpkg.cmake"
cmake --build build -j$(nproc)

- name: Run Tests
working-directory: heiFIP
run: |
ctest --output-on-failure
8 changes: 7 additions & 1 deletion .github/workflows/build_test_macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,15 @@ jobs:
brew install pkg-config cmake git

- name: Configure and Build
working-directory: heiFIP
run: |
cd heiFIP/
cmake -B build -S . \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_TOOLCHAIN_FILE="$VCPKG_INSTALLATION_ROOT/scripts/buildsystems/vcpkg.cmake"
cmake --build build -j$(sysctl -n hw.ncpu)

- name: Run Tests
working-directory: heiFIP
run: |
cd build
ctest --output-on-failure
188 changes: 94 additions & 94 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,102 +83,102 @@ The idea to create heiFIP came from working with Deep Learning approaches to cla

| Image Type | Description | Example |
|------------|-------------|---------|
| Packet | Converts a single packet into a square image. Size depends on the total length | ![SMB Connection](https://raw.githubusercontent.com/stefanDeveloper/heiFIP/heiFIP-cpp/examples/packet.png?raw=true) |
| Flow | Converts a flow packet into a square image | ![SMB Connection](https://raw.githubusercontent.com/stefanDeveloper/heiFIP/heiFIP-cpp/examples/flow-tiled.png?raw=true) |
| Markov Transition Matrix Packet | Converts a packet into a Markov Transition Matrix. Size is fixed to 16x16. | ![SMB Connection](https://raw.githubusercontent.com/stefanDeveloper/heiFIP/heiFIP-cpp/examples/markov-packet.png?raw=true) |
| Markov Transition Matrix Flow | Converts a flow into a Markov Transition Matrix. It squares the image based on the number of packets | ![SMB Connection](https://raw.githubusercontent.com/stefanDeveloper/heiFIP/heiFIP-cpp/examples/markov-flow.png?raw=true) |
| Packet | Converts a single packet into a square image. Size depends on the total length | ![SMB Connection](https://raw.githubusercontent.com/stefanDeveloper/heiFIP/main/examples/packet.png?raw=true) |
| Flow | Converts a flow packet into a square image | ![SMB Connection](https://raw.githubusercontent.com/stefanDeveloper/heiFIP/main/examples/flow-tiled.png?raw=true) |
| Markov Transition Matrix Packet | Converts a packet into a Markov Transition Matrix. Size is fixed to 16x16. | ![SMB Connection](https://raw.githubusercontent.com/stefanDeveloper/heiFIP/main/examples/markov-packet.png?raw=true) |
| Markov Transition Matrix Flow | Converts a flow into a Markov Transition Matrix. It squares the image based on the number of packets | ![SMB Connection](https://raw.githubusercontent.com/stefanDeveloper/heiFIP/main/examples/markov-flow.png?raw=true) |

## Requirements

* **C++ Compiler**: GCC ≥ 9.0, Clang ≥ 10, or MSVC 2019 with C++17 support.
* **CMake**: Version ≥ 3.14
* **vcpkg**: A C++ package manager to automatically download and build dependencies.

Dependencies managed automatically by `vcpkg`:
* **PcapPlusPlus**
* **OpenSSL**
* **OpenCV**
* **libpcap** (Linux/macOS) / `pthread`

## Building from source

We use `vcpkg` to manage all C++ dependencies for heiFIP smoothly. If you don't have `vcpkg` installed, follow their [official instructions](https://github.com/microsoft/vcpkg#quick-start-windows-linux-macos).

Ensure the `VCPKG_ROOT` environment variable is set to your `vcpkg` installation path (e.g., `export VCPKG_ROOT=~/vcpkg`).

```bash
# Clone this repo
git clone https://github.com/stefanDeveloper/heiFIP.git
cd heiFIP/heiFIP/

# Set up vcpkg
git clone https://github.com/microsoft/vcpkg.git
./vcpkg/bootstrap-vcpkg.sh
export VCPKG_ROOT=$(pwd)/vcpkg

# Create build directory and run CMake using the vcpkg toolchain
# The toolchain will automatically read vcpkg.json and install dependencies!
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake"

# Compile the project
cmake --build build -j$(nproc)

# The executables 'heiFIP' and 'main' will be produced in build/
```


## Getting Started

After installation the command line interface can be used to extract images from pcap files witht he following command
```bash
./heiFIPCpp \
--name HelloHeiFIP
--input /path/to/capture.pcap \
--output /path/to/outdir \
--threads 4 \
--processor HEADER \
--mode FlowImageTiledAuto \
--dim 16 \
--apppend \
--fill 0 \
--min-dim 10 \
--max-dim 2000 \
--min-pkts 10 \
--max-pkts 100 \
--remove-dup
```

### Options
| Flag | Description |
| ------------------- | -------------------------------------------------------------- |
| `-i`, `--input` | Input PCAP file path |
| `-o`, `--output` | Output directory |
| `-t`, `--threads` | Number of worker threads (default: 1) |
| `-p`, `--processor` | Preprocessing: `NONE` or `HEADER` |
| `-m`, `--mode` | Image type: `PacketImage`, `FlowImage`, `FlowImageTiledFixed`, |
| | `FlowImageTiledAuto`, `MarkovTransitionMatrixFlow`, |
| | `MarkovTransitionMatrixPacket` |
| `--dim` | Base dimension for image (e.g. width/height in pixels) |
| `--fill` | Fill or padding value (0–255) |
| `--cols` | Number of columns (for tiled/fixed or Markov flow) |
| `--auto-dim` | Enable auto‑dimension selection (bool) |
| `--append` | Enable auto‑dimension selection (bool) |
| `--min-dim` | Minimum allowed image dimension |
| `--max-dim` | Maximum allowed image dimension |
| `--min-pkts` | Minimum packets per flow (for tiled/flow modes) |
| `--max-pkts` | Maximum packets per flow |
| `--remove-dup` | Remove duplicate flows/packets by hash |
| `--name` | Filname of processed image |
| `-h`, `--help` | Show this help message |

## Extending

To add a new image type:

1. Define a new `ImageArgs` struct in `extractor.cpp`.
2. Extend the `ImageType` enum.
3. Implement the conversion in `PacketProcessor::createImageFromPacket()`.
4. Update the CLI `--mode` parser to include your new type.
* **C++ Compiler**: GCC ≥ 9.0, Clang ≥ 10, or MSVC 2019 with C++20 support.
* **CMake**: Version ≥ 3.14
* **vcpkg**: A C++ package manager to automatically download and build dependencies.
Dependencies managed automatically by `vcpkg`:
* **PcapPlusPlus**
* **OpenSSL**
* **OpenCV**
* **GTest** (GoogleTest)
* **libpcap** (Linux/macOS) / `pthread`
## Building from source
We use `vcpkg` to manage all C++ dependencies for heiFIP smoothly. If you don't have `vcpkg` installed, follow their [official instructions](https://github.com/microsoft/vcpkg#quick-start-windows-linux-macos).
Ensure the `VCPKG_ROOT` environment variable is set to your `vcpkg` installation path (e.g., `export VCPKG_ROOT=~/vcpkg`).
```bash
# Clone this repo
git clone https://github.com/stefanDeveloper/heiFIP.git
cd heiFIP/heiFIP/
# Create build directory and run CMake using the vcpkg toolchain
# The toolchain will automatically read vcpkg.json and install dependencies!
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake"
# Compile the project
cmake --build build -j$(nproc)
# The executables 'heiFIP' and 'main' will be produced in build/
```
## Testing
heiFIP includes a comprehensive test suite covering protocol layers, image generation, and CLI functionality.
```bash
# Run all tests using CTest
cd build
ctest --output-on-failure
```
## Getting Started

After installation the command line interface can be used to extract images from pcap files with the following command (Note: `--dim` defaults to 16 and `--processor` to `NONE`).
```bash
./build/heiFIP \
--name HelloHeiFIP \
--input /path/to/capture.pcap \
--output /path/to/outdir \
--threads 4 \
--processor HEADER \
--mode FlowImageTiledAuto \
--max-pkts 100 \
--remove-dup
```
### Options
| Flag | Description |
| ------------------- | -------------------------------------------------------------- |
| `-i`, `--input` | Input PCAP file path |
| `-o`, `--output` | Output directory |
| `-t`, `--threads` | Number of worker threads (default: 1) |
| `-p`, `--processor` | Preprocessing: `NONE` or `HEADER` (default: `NONE`) |
| `-m`, `--mode` | Image type: `PacketImage`, `FlowImage`, `FlowImageTiledFixed`, |
| | `FlowImageTiledAuto`, `MarkovTransitionMatrixFlow`, |
| | `MarkovTransitionMatrixPacket` |
| `--dim` | Base dimension for image (default: 16) |
| `--fill` | Fill or padding value (0–255) |
| `--cols` | Number of columns (for tiled/fixed or Markov flow) |
| `--auto-dim` | Enable auto‑dimension selection |
| `--append` | Enable append mode for FlowImage |
| `--min-dim` | Minimum allowed image dimension |
| `--max-dim` | Maximum allowed image dimension |
| `--min-pkts` | Minimum packets per flow (for tiled/flow modes) |
| `--max-pkts` | Maximum packets per flow |
| `--remove-dup` | Remove duplicate flows/packets by hash |
| `--name` | Filename of processed image |
| `-h`, `--help` | Show this help message |
## Extending
To add a new image type:
1. Define a new `ImageArgs` struct in `extractor.hpp`.
2. Extend the `ImageType` enum and the `ImageArgsVariant` in `extractor.hpp`.
3. Implement the image logic (following the pattern of `FlowImage` or `heiFIPPacketImage`).
4. Add the image creation case in `FIPExtractor::createMatrix()`.
5. Update the CLI `--mode` parser in `cli.cpp` to include your new type.

---

Expand Down
56 changes: 40 additions & 16 deletions heiFIP/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,38 +17,62 @@ set(CMAKE_C_FLAGS_RELEASE "-O3 -march=native -funroll-loops -flto=auto -DNDEBU
find_package(OpenSSL REQUIRED)
find_package(OpenCV REQUIRED)
find_package(PcapPlusPlus CONFIG REQUIRED)
find_package(GTest CONFIG REQUIRED)

# === 4. User project headers ===
include_directories(
${CMAKE_SOURCE_DIR}
${CMAKE_SOURCE_DIR}/assets
${CMAKE_SOURCE_DIR}/plugins
${CMAKE_SOURCE_DIR}/images
${CMAKE_SOURCE_DIR}/layers
)

# === 5. Executables ===
# === 5. Enable Testing ===
enable_testing()

# === 6. Common Libraries ===
set(HEIFIP_LIBS
OpenSSL::Crypto
OpenSSL::SSL
PcapPlusPlus::Common++
PcapPlusPlus::Packet++
PcapPlusPlus::Pcap++
${OpenCV_LIBS}
)

# === 7. Executables ===
add_executable(heiFIP cli.cpp)
add_executable(main main.cpp)
# main2.cpp removed

# === 6. Apply libraries ===
foreach(_target IN ITEMS heiFIP main)
target_link_libraries(${_target} PUBLIC
OpenSSL::Crypto
OpenSSL::SSL
PcapPlusPlus::Common++
PcapPlusPlus::Packet++
PcapPlusPlus::Pcap++
${OpenCV_LIBS}
)

# === 8. Tests ===
add_executable(heiFIP_tests
tests/test_packet_image.cpp
tests/test_flow_image.cpp
tests/test_tiled_images.cpp
tests/test_markov.cpp
tests/test_tile_utils.cpp
tests/test_extractor.cpp
tests/test_layers.cpp
tests/test_cli.cpp
)

# === 9. Apply libraries ===
foreach(_target IN ITEMS heiFIP main heiFIP_tests)
target_link_libraries(${_target} PUBLIC ${HEIFIP_LIBS})
endforeach()

# === 7. Optimization and LTO in Release builds ===
foreach(_target IN ITEMS heiFIP main)
target_link_libraries(heiFIP_tests PUBLIC GTest::gtest_main)

# === 10. Optimization and LTO in Release builds ===
foreach(_target IN ITEMS heiFIP main heiFIP_tests)
target_compile_options(${_target} PUBLIC
$<$<CONFIG:Release>:-O3 -march=native -funroll-loops -flto=auto -DNDEBUG>
)
target_link_options(${_target} PUBLIC
$<$<CONFIG:Release>:-flto=auto>
)
endforeach()
endforeach()

include(GoogleTest)
gtest_discover_tests(heiFIP_tests)
Loading
Loading