Adaptive AI Task Partitioning and Safe Offloading in Heterogeneous Edge-Cloud Continuum

Profiling Neural Network Partitioning Strategies for Inference across the Computing Continuum

As deep learning permeates latency-sensitive domains such as autonomous driving and smart surveillance, deploying neural networks (NNs) across the computing continuum (CC), from IoT devices to edge servers and cloud platforms, has become increasingly important. In such heterogeneous IoT-Edge-Cloud environments, distributed inference promises reduced latency, improved privacy, and better resource utilization. Yet, determining how to deploy NNs over heterogeneous IoT-Edge-Cloud nodes remains a difficult and largely manual process. This paper presents a principled and extensible framework for evaluating distributed inference of NNs in heterogeneous CC infrastructures. We introduce a formal model that unifies functional, pipelined, and data-parallel partitioning strategies within a single abstraction over heterogeneous CC topologies, enabling structured cross-strategy comparison. Building on this foundation, we implement a distributed inference orchestrator that supports flexible deployment of partitioned CNNs, and introduce PartiBench, a benchmarking tool that profiles segments and guides their placement. Our evaluation demonstrates how the framework exposes key performance trade-offs, offering actionable insights into latency, memory use, and communication overhead across IoT-Edge-Cloud nodes. These contributions enable empirical, cross-strategy comparison of distributed inference deployments and provide a basis for future automated placement methods in heterogeneous IoT-Edge-Cloud systems.

Dependency-aware adaptive task offloading in edge-AI IoT networks using transformer-DDQN

Integrating AI Clusters into Virtual Private Cloud

While commodity NIC-based back-end AI networks offer ultra-high intra-cluster bandwidth for distributed training, their limited programmability and on-chip resources hinder the implementation of advanced VPC features such as fine-grained isolation and stateful security policies. Furthermore, access to resources within the VPC needs to be routed through the front-end DPU, which is shared by the scale-up domain. The mismatch between the front-end DPU’s bandwidth and the back-end requirements causes GPU underutilization when intensive VPC communication is required for content recommendation, AIGC, and federated learning workloads. We propose an architecture that decouples complex policy enforcement from high-speed packet forwarding to support VPC semantics on back-end NICs and enable front-end/back-end integration. Evaluations show near-full GPU utilization in our analytical model and 71 μ s P999 extra latency of the first packet, suggesting that commodity hardware can support both high-throughput AI training and flexible VPC features.

A Resource-Aware Asynchronous PBT Framework with Dynamic LoRA for Heterogeneous Edge Clusters

Traditionally, training and hyperparameter optimization of deep neural networks rely heavily on centralized cloud data centers. However, privacy concerns are driving a paradigm shift to move model fine-tuning directly to localized edge environments. Migrating Population-Based Training (PBT) to the edge presents severe challenges: unlike uniform cloud servers, edge networks exhibit extreme hardware heterogeneity. While Low-Rank Adaptation (LoRA) accelerates individual trials on weak devices, permanently maintaining adapters causes structural divergence during PBT weight inheritance. To cope with these edge-specific constraints, we propose Dynamic LoRA-PBT, an asynchronous hardware-software co-design. Systematically, it mitigates extreme evolutionary staleness via a capability-aware scheduler and a late-stage CPU dropout mechanism. Algorithmically, it introduces a Merge-and-Unload strategy, injecting LoRA for early exploration and explicitly merging it into the dense model before mutation. Preliminary evaluations on an 11-node CPU/GPU edge-server cluster demonstrate a 23.4 percent reduction in Time-to-Accuracy (TTA). As an ongoing work, we are currently integrating Transformer architectures and conducting rigorous statistical validations to solidify this framework.

Heterogeneous Low-Power Agentic Architecture for Edge Distributed Home Surveillance

Recent advancements in edge hardware enabled increasingly complex artificial intelligence workloads to be executed directly on resource-constrained devices. However, single-edge devices remain computationally limited compared with hybrid or cloud alternatives. This work presents a fully edge-based architecture implementing agentic AI on heterogeneous devices for home surveillance with a natural language interface, in which analysis tasks are parallelized across the distributed devices. The system is tested on a two-node prototype: a Raspberry Pi 5 with a Hailo-10H accelerator and an STM32MP257FDK board. Dynamic leader election allows recovery from the loss of any single node, including the leader, with graceful degradation. Qwen3-1.7B achieved 93% tool-selection accuracy over 24 tools and maintained robust summarization performance, outperforming FunctionGemma-270M. The jina-reranker-v2-base-multilingual cross-encoder performs comparably to the tested LLMs at tool selection on small pool sizes (i.e., with few tools to choose from). Hardware accelerators improved inference energy efficiency by 2.0× to 17.3×, while leader execution incurred less than 2 W of additional power. The main bottleneck is model latency rather than network overhead, and KV-cache reuse proved essential to meet the latency constraint on the STM32MP257FDK. The proposed prototype coordinates inference across heterogeneous edge devices to answer queries, with the weaker node exhibiting scalability limitations.

Edge–Cloud Collaborative Deep Learning Framework for Real-Time IoT Analytics

The Internet of Things (IoT) ecosystem has expanded and produced massive volumes of real-time streaming data, that must be processed quickly and in an efficient way. The latency problem, bandwidth congestion and bandwidth limitations in the traditional cloud-centric processing architecture are significant challenges, particularly in latency-sensitive systems such as smart healthcare, industrial automation and intelligent transportation systems. While edge computing reduces the amount of data that needs to be sent, edge devices are typically more limited in resources, which means they cannot run complex deep learning models. To address these limitations, this paper proposes a new Edge–Cloud Collaborative Deep Learning (ECC-DL) Framework to orchestrate different aspects of inference and learning between the edge nodes and cloud server. The lightweight convolutional neural networks (CNNs) are deployed at the edge nodes to extract features and make initial inferences, and the heavier end models are deployed at the cloud end for global analysis and regular retraining. To optimize the partitioning of the workload between the resources, an adaptive workload partitioning algorithm is designed that considers latency, bandwidth and computational resources. By experimental evaluation, latency is reduced by 40% and bandwidth consumption is reduced by 30% and prediction accuracy is improved over the centralized approach.