Search

The cloud server stores its unique DE-duplicate data replication cloud storage system as outsourced only data copies. Deduplication cloud storage audits can verify the integrity of data stored in the cloud, while the cloud calculates the need to maintain only one copy of the copied file. The existing de-duplication auditing solutions for cloud storage are vulnerable to brute-force dictionary attacks, which is a disclosure leakage of user privacy and more time consumption to find a duplicate files in cloud. Users must check the integrity of lightweight cloud computing data to create data authentication and retrieve files in the cloud. The proposed system uses a secure Array Preserving Triple Data Encryption (AP3DE) algorithm to verify with confidence the expertise of aggregate technology. In this method to analysis the user upload document before store on cloud. The aggregator to provide an authentication access the data access role to cloud user. Cloud protects the privacy, confidentiality, and integrity of data stored in the cloud by generating the code of the authentication file and the encryption key of the file through the proposed system test. The Index-level Boundary Pattern Convergent Encryption (IBPCE) algorithm integrates dynamic data handling and data exclusion capabilities for secure cloud storage privacy security public audits. The cloud data security using a Convergent international data encryption algorithm used to encrypted data. It is a 64 bit plaintext and cipher text blocks is controlled by 128 bit key. Analysis of safety and efficiency shows that the proposed mechanism is highly efficient and safe. In this proposed IBPCE method to manage and reduce the cloud storage compare to other existing method.

Data deduplication can efficiently eliminate data redundancies in cloud storage and reduce the bandwidth requirement of users. However, most previous schemes depending on the help of a trusted key server (KS) are vulnerable and limited because they suffer from revealing information, poor resistance to attacks, great computational overhead, etc. In particular, if the trusted KS fails, the whole system stops working, i.e., single-point-of-failure. In this paper, we propose a Secure and Efficient data Deduplication scheme (named SED) in a JointCloud storage system which provides the global services via collaboration with various clouds. SED also supports dynamic data update and sharing without the help of the trusted KS. Moreover, SED can overcome the single-point-of-failure that commonly occurs in the classic cloud storage system. According to the theoretical analyses, our SED ensures the semantic security in the random oracle model and has strong anti-attack ability such as the brute-force attack resistance and the collusion attack resistance. Besides, SED can effectively eliminate data redundancies with low computational complexity and communication and storage overhead. The efficiency and functionality of SED improves the usability in client-side. Finally, the comparing results show that the performance of our scheme is superior to that of the existing schemes.

Cloud storage is a cost-effective platform to accommodate massive data at low cost. However, advances of cloud services propel data generation, which pushes storage servers to its limit. Deduplication is a popular technique enjoyed by most current cloud servers, which detects and deletes redundant data to save storage and bandwidth. For security concerns, proof-of-ownership (PoW) can be used to guarantee ownership of data such that no malicious user could pass deduplication easily or utilize such mechanism for malicious purposes. Generally, PoW is implemented in static data archive where the data file is supposed to be read-only. However, to satisfy users' needs for dynamical manipulation on data and support real-time data services, it is required to devise efficient PoW for dynamic archive. In this work, we propose the notion of bidirectional and malleable proof-of-ownership (BM-PoW) for the above challenge. Our proposed BM-PoW consists of bidirectional PoW (B-PoW), malleable PoW (M-PoW) and dispute arbitration protocol DAP. We provide the security analysis of our proposal, and performance evaluation that suggests our proposed B-PoW is secure and efficient for large file in static data archive. In addition, our proposed M-PoW achieves acceptable performance under dynamic setting where data is supposed to be outsourced first and updated later in dynamic data archive.

The pervasive deployment of cloud services poses an ever-increasing demand for cross-client deduplication solutions to save network bandwidth, lower storage costs, and improve backup speeds. However, existing solutions typically depend on lock based approaches relying on a centralized chunk database, which tends to hinder performance scalability. In this work, we present a new cross-client cloud backup solution, named Duplicacy, based on a Lock-Free Deduplication approach. Lock-Free Deduplication stores chunks to network or cloud storage using content hashes as file names. It then adopts a two-step fossil deletion algorithm to solve the hard problem of deleting unreferenced chunks in the presence of concurrent backups, without the need for any locks. Experiments demonstrate that Duplicacy enables significant performance improvement for backups over previous well-known backup tools. In addition, Duplicacy can work with many general-purpose network or cloud storage services which only support a basic set of file operations, and turn them into sophisticated deduplication-aware storage servers without server-side changes.

Data deduplication technique could greatly save the storage overhead of the cloud by eliminating duplicated data and retaining one copy. In order to ensure the privacy of the data against an untrusted cloud, many cryptographic schemes have been proposed to make deduplication feasible in ciphertext. A typical scheme is Message-Locked Encryption (MLE), which takes cryptographic hash values of messages as encryption keys. However, MLE is vulnerable to side-channel attacks. Current solutions trying to mitigate these attacks raise either expensive overhead or security drawbacks. In this paper, we propose a secure data deduplication system against an untrusted cloud with resistance to two typical side-channel attacks, namely probe attack and key-cache attack. Our system uses fog computing to devise two new techniques to solve the two side-channel attacks with new security and efficiency trade-offs. The analysis and evaluation show that our system achieves better trade-offs compared with previous works.

Automated detection of software vulnerabilities is a fundamental problem in software security. Existing program analysis techniques either suffer from high false positives or false negatives. Recent progress in Deep Learning (DL) has resulted in a surge of interest in applying DL for automated vulnerability detection. Several recent studies have demonstrated promising results achieving an accuracy of up to 95% at detecting vulnerabilities. In this paper, we ask, "how well do the state-of-the-art DL-based techniques perform in a real-world vulnerability prediction scenario?". To our surprise, we find that their performance drops by more than 50%. A systematic investigation of what causes such precipitous performance drop reveals that existing DL-based vulnerability prediction approaches suffer from challenges with the training data (e.g., data duplication, unrealistic distribution of vulnerable classes, etc.) and with the model choices (e.g., simple token-based models). As a result, these approaches often do not learn features related to the actual cause of the vulnerabilities. Instead, they learn unrelated artifacts from the dataset (e.g., specific variable/function names, etc.). Leveraging these empirical findings, we demonstrate how a more principled approach to data collection and model design, based on realistic settings of vulnerability prediction, can lead to better solutions. The resulting tools perform significantly better than the studied baseline: up to 33.57% boost in precision and 128.38% boost in recall compared to the best performing model in the literature. Overall, this paper elucidates existing DL-based vulnerability prediction systems' potential issues and draws a roadmap for future DL-based vulnerability prediction research. In that spirit, we make available all the artifacts supporting our results: https://git.io/Jf6IA., Comment: Under Review IEEE Transactions on Software Engineering

The healthcare industry faces challenges regarding the security and efficiency of data management for patient health records (PHRs). We propose SafePHR: a secure and efficient medical data service with application aware deduplication in fog-to-multicloud encrypted storage. It introduces a fog-to-multicloud cooperative storage model, which combines low-latency and high-safety local fog with unlimited capacity and built-in disaster recovery in remote multi-cloud to enhance eHealth data management. It also provides an application aware secure deduplication scheme to improve the data traffic and space efficiency of cloud storage for encrypted PHRs using variants of convergent encryption. Then, it further enhances the resiliency and security of cloud storage by striping encrypted data across multiple cloud vendors with a fault-tolerant coding scheme. Compared with the state-of-the-art of cloud-assisted eHealth systems, our experiments demonstrate that SafePHR can ensure data confidentiality of deduplication, achieve a competitive low cloud storage space overhead, and improve system performance with high fault-tolerance capability.

Data reduction has gained growing emphasis due to the rapidly unsystematic increase in digital data and has become a sensible approach to big data systems. Data deduplication is a technique to optimize the storage requirements and plays a vital role to eliminate redundancy in large-scale storage. Although it is robust in finding suitable chunk-level break-points for redundancy elimination, it faces key problems of (1) low chunking performance, which causes chunking stage bottleneck, (2) a large variation in chunk-size that reduces the efficiency of deduplication, and (3) hash computing overhead. To handle these challenges, this paper proposes a technique for finding proper cut-points among chunks using a set of commonly repeated patterns (CRP), it picks out the most frequent sequences of adjacent bytes (i.e., contiguous segments of bytes) as breakpoints. Besides to scalable lightweight triple-leveled hashing function (LT-LH) is proposed, to mitigate the cost of hashing function processing and storage overhead; the number of hash levels used in the tests was three, these numbers depend on the size of data to be de-duplicated. To evaluate the performance of the proposed technique, a set of tests was conducted to analyze the dataset characteristics in order to choose the near-optimal length of bytes used as divisors to produce chunks. Besides this, the performance assessment includes determining the proper system parameter values leading to an enhanced deduplication ratio and reduces the system resources needed for data deduplication. Since the conducted results demonstrated the effectiveness of the CRP algorithm is 15 times faster than the basic sliding window (BSW) and about 10 times faster than two thresholds two divisors (TTTD). The proposed LT-LH is faster five times than Secure Hash Algorithm 1 (SHA1) and Message-Digest Algorithm 5 (MD5) with better storage saving.

In this paper, we propose SPADE, an encrypted data deduplication scheme that resists compromised key servers and frees users from the key management problem. Specifically, we propose a proactivization mechanism for the servers-aided message-locked encryption (MLE) to periodically substitute key servers with newly employed ones, which renews the security protection and retains encrypted data deduplication. We present a servers-aided password-hardening protocol to resist dictionary guessing attacks. Based on the protocol, we further propose a password-based layered encryption mechanism and a password-based authentication mechanism and integrate them into SPADE to enable users to access their data only using their passwords. Provable security and high efficiency of SPADE are demonstrated by comprehensive analyses and experimental evaluations.

It is observed that modern vehicles are becoming more and more powerful in computing, communications, and storage capacity. By interacting with other vehicles or with local infrastructures (i.e., fog) such as road-side units, vehicles and fog devices can collaboratively provide services like crowdsensing in an efficient and secure way. Unfortunately, it is hard to develop a secure and privacy-preserving crowdsensing report deduplication mechanism in such a system. In this paper, we propose a scheme FVC-Dedup to address this challenge. Specifically, we develop cryptographic primitives to realize secure task allocation and guarantee the confidentiality of crowdsensing reports. Duringthe report submission, we improve the message-lock encryption (MLE) scheme to realize privacy-preserving report deduplication and resist the fake duplicate attacks. Besides, we construct a novel signature scheme to achieve efficient signature aggregation and record the contributions of each participant fairly without knowing the crowdsensing data. The security analysis and performance evaluation demonstrate that FVC-Dedup can achieve secure and privacy-preserving report deduplication with moderate computing and communication overhead.

Non-Volatile Memory (NVM) offers byte-addressability and persistency Because NVM can be plugged into memory and provide low latency, it offers a new opportunity to build new database systems with a single-layer storage design. A single-layer NVM-Native DataBase (N2DB) provides zero copy and log freedom. Hence, all data are stored in NVM and there is no extra data duplication and logging during execution. N2DB avoids complex data synchronization and logging overhead in the two-layer storage design of disk-oriented databases and in-memory databases. Garbage Collection (GC) is critical in such an NVM-based database because memory leaks on NVM are durable. Moreover, data recovery is equally essential to guarantee atomicity, consistency, isolation, and durability properties. Without logging, it is a great challenge for N2DB to restore data to a consistent state after crashes and recoveries. This paper presents the GC and data recovery mechanisms for N2DB. Evaluations show that the overall performance of N2DB is up to 3.6× higher than that of InnoDB. Enabling GC reduces performance by up to 10%, but saves storage space by up to 67%. Moreover, our data recovery requires only 0.2% of the time and half of the storage space of InnoDB.

Message-Locked Encryption (MLE) is a useful tool to enable deduplication over encrypted data in cloud storage. It can significantly improve the cloud service quality by eliminating redundancy to save storage resources, and hence user cost, and also providing defense against different types of attacks, such as duplicate faking attack and brute-force attack. A typical MLE scheme only focuses on deduplication. On the other hand, supporting search operations on stored content is another essential requirement for cloud storage. In this paper, we present a message-locked searchable encryption (MLSE) scheme in a dual-server setting, which achieves simultaneously the desirable features of supporting deduplication and enabling users to perform search operations over encrypted data. In addition, it supports both multi-keyword and negative keyword searches. We formulate the security notions of MLSE and prove our scheme satisfies all the security requirements. Moreover, we provide an interesting extension of our construction to support Proof of Storage (PoS). Compared with the existing solutions, MLSE achieves better functionalities and efficiency, and hence enables more versatile and efficient cloud storage service.

As a novel network infrastructure that realizes the interconnection of humans, machines, and things, the 5G-based or blockchain-based applications have been widely deployed in the Industrial Internet of Things (IIoT). However, there are still many challenges to be solved, such as poor scalability, low efficiency, and privacy leakages. In the special issue, we present eight advanced solutions in data analysis, secure communication, authentication, access control, and data deduplication.

Encrypted deduplication combines encryption and deduplication in a seamless way to provide confidentiality guarantees for the physical data in deduplicated storage, yet it incurs substantial metadata storage overhead due to the additional storage of keys. We present a new encrypted deduplication storage system called Metadedup, which suppresses metadata storage by also applying deduplication to metadata. Its idea builds on indirection, which adds another level of metadata chunks that record metadata information. We find that metadata chunks are highly redundant in real-world workloads and hence can be effectively deduplicated. We further extend Metadedup to incorporate multiple servers via a distributed key management approach, so as to provide both fault-tolerant storage and security gaurantees. We extensively evaluate Metadedup from performance and storage efficiency perspectives. We show that Metadedup achieves high throughput in writing and restoring files, and saves the metadata storage by up to 93.94% for real-world backup workloads.

Mobile crowdsensing provides the data collection and sharing for 5G-enabled industrial Internet of Things. However, the redundant and duplicated heterogeneous sensing data brings unnecessary heavy storage and communication overhead. In this paper, we propose a secure heterogeneous data deduplication scheme, which introduces the privacy-preserving cosine similarity computing to eliminate the replicate sensing data without privacy leakage in mobile crowdsensing. Specifically, we use the proxy re-encryption algorithm to realize secure and accurate tasks assignment via fog-assisted. Based on lightweight two-party random masking and polynomial aggregation techniques, we achieve privacy-preserving cosine similarity computing protocol. Finally, we conduct the privacy analysis, and experiment results on real-world dataset shows that our approach is practical and effective.

Secure data deduplication can significantly reduce the communication and storage overheads in cloud storage services, and has potential applications in our big data-driven society. Existing data deduplication schemes are generally designed to either resist brute-force attacks or ensure the efficiency and data availability, but not both conditions. We are also not aware of any existing scheme that achieves accountability, in the sense of reducing duplicate information disclosure (e.g., to determine whether plaintexts of two encrypted messages are identical). In this paper, we investigate a three-tier cross-domain architecture, and propose an efficient and privacy-preserving big data deduplication in cloud storage (hereafter referred to as EPCDD). EPCDD achieves both privacy-preserving and data availability, and resists brute-force attacks. In addition, we take accountability into consideration to offer better privacy assurances than existing schemes. We then demonstrate that EPCDD outperforms existing competing schemes, in terms of computation, communication and storage overheads. In addition, the time complexity of duplicate search in EPCDD is logarithmic.

Traditional band selection methods only analyze one data set at a time, and start searching band subsets from the zero ground state of knowledge, which can not effectively mine spectral information to guide band selection. However, for hyperspectral images obtained by the same sensor, the spectral information has similar physical meaning (radiance or reflectivity). Collaborative analysis technology can analyze multiple hyperspectral datasets to explore the inherent spectral features shared among them. In this paper, a multiple data sets collaborative analysis framework for hyperspectral band selection is proposed to realize spectral information communication, thereby guiding and promoting band selection of each data set. Different band selection tasks are established pertinently, then the evolutionary multitasking band selection method is designed to facilitate the knowledge sharing of different band selection tasks. More importantly, the interaction mechanism among different data sets is adjusted dynamically, thereby improving the cooperation ability of the collaborative analysis framework. Besides, a predominant gene reservation crossover and a deduplication mutation are designed for retaining the promising bands and avoiding the selection of repeat bands. Experiments indicate that the proposed collaborative analysis method works more efficiently than the comparison methods and successfully enhances accuracy and convergence compared to single data set analysis.

Cloud storage as one of the most important services of cloud computing facilitates cloud users to outsource their data to the cloud for storage and share them with authorized users. In cloud storage, secure deduplication has been investigated as it can eliminate the redundancy over the encrypted data to reduce storage space and communication overhead. Regarding the security and privacy, many existing secure deduplication schemes generally focus on achieving the following properties: data confidentiality, tag consistency, access control and resistance to brute force attacks. However, as far as we know, none of them can achieve these four requirements at the same time. To overcome this shortcoming, in this paper, we propose an efficient secure deduplication scheme that supports user-defined access control. Specifically, by allowing only the cloud service provider to authorize data access on behalf of data owners, our scheme can maximally eliminate duplicates without violating the security and privacy of cloud users. Detailed security analysis shows that our authorized secure deduplication scheme achieves data confidentiality and tag consistency while resisting brute-force attacks. Furthermore, extensive simulations demonstrate that our scheme outperforms the existing competing schemes, in terms of computational, communication and storage overheads as well as the effectiveness of deduplication.

Data deduplication technique has been widely adopted by commercial cloud storage providers, which is important in coping with the explosive growth of data. To further protect the security of users' sensitive data in the outsourced storage mode, many secure data deduplication schemes have been designed and applied in various scenarios. Among these schemes, secure and efficient re-encryption for encrypted data deduplication attracted the attention of many scholars, and many solutions have been designed to support dynamic ownership management. In this paper, we focus on the re-encryption deduplication storage system and show that the recently designed lightweight rekeying-aware encrypted deduplication scheme is vulnerable to the stub-reserved attack. Furthermore, we propose a secure data deduplication scheme with efficient re-encryption based on the convergent all-or-nothing-transform (CAONT) and randomly sampled bits from the Bloom filter. Due to the property of hash function, our scheme can resist the stub-reserved attack and guarantee the data privacy of data owners' sensitive data. Moreover, instead of re-encrypting the entire package, data owners are only required to re-encrypt a small part of it through the CAONT, thereby effectively reducing the computation overhead. Finally, security analysis and experimental results show that our scheme is secure and efficient in re-encryption.

Deduplication methods were designed to destroy copy information which bring about capacity of single duplicates of information as it were. Information Deduplication diminishes the circle space needed to store the back-ups in the extra room, tracks and kill the second duplicate of information inside the capacity unit. It permits as it were one case information event to be put away initially and afterward following occasions will be given reference pointer to the first information put away. In a Big information stockpiling climate, immense measure of information should be secure. For this legitimate administration, work, misrepresentation identification, investigation of information protection is an significant theme to be thought of. This paper inspects and assesses the common deduplication procedures and which are introduced in plain structure. In this review, it was seen that the secrecy and security of information has been undermined at numerous levels in common strategies for deduplication. Albeit much exploration is being done in different zones of distributed computing still work relating to this point is inadequate. To get rid of duplicate data which results in storage of single copies of data, data deduplication techniques were used. Data deduplication helps in decreasing storage capacity requirements and eliminates extra copies of same data inside storage unit. Proper management, work, fraud detection, analysis of data privacy are the topics to be considered in a big data storage environment, since, large amount of data needs to be secure. At many levels in general techniques for deduplication it is observed that safety of data and confidentiality has been compromised. Even though more research is being carried out in different areas of cloud computing still work related to this topic is little.

Surveillance cameras have been extensively used in smart cities and high security zones. However, with the exploding deployment of smart cameras, the rapid growth of cloud workloads from vision-based IoT applications are becoming a huge burden for all cloud service providers. Some researchers have proposed mechanisms, such as compression and deduplication to reduce the video traffic size, but these methods cannot offset the enormous growth of data volume. Most of the surveillance video data do not need to be proceeded in real time. By making use of the IoT camera’s onboard resources to store the data, the cloud workloads can be fundamentally reduced. However, recent incidents have posed a new, powerful geo-range attack, where the attacker may compromise a group of surveillance cameras located within an area. Existing simple onboard solutions cannot offer secure defense against such geo-range attacks. To tackle the problem, we develop Centipede , a cooperative video data storage system that distributes video content across geographically dispersed surveillance cameras. It generates secure copies for the video content and enhances data security by judiciously distributing erasure-coded video blocks across optimally-chosen surveillance cameras. In this article, we implement Centipede and evaluate its performance. Centipede is the first solution that can fundamentally reduce the cloud workload and defend against geo-range attacks.

Data deduplication techniques construct an index consisting of fingerprint entries to identify and eliminate duplicated copies of repeating data. The bottleneck of disk-based index lookup and data fragmentation caused by eliminating duplicated chunks are two challenging issues in data deduplication. Deduplication-based backup systems generally employ containers storing contiguous chunks together with their fingerprints to preserve data locality for alleviating the two issues, which is still inadequate. To address these two issues, we propose a container utilization based hot fingerprint entry distilling strategy to improve the performance of deduplication-based backup systems. We divide the index into three parts: hot fingerprint entries, fragmented fingerprint entries, and useless fingerprint entries. A container with utilization smaller than a given threshold is called a sparse container . Fingerprint entries that point to non-sparse containers are hot fingerprint entries. For the remaining fingerprint entries, if a fingerprint entry matches any fingerprint of forthcoming backup chunks, it is classified as a fragmented fingerprint entry. Otherwise, it is classified as a useless fingerprint entry. We observe that hot fingerprint entries account for a small part of the index, whereas the remaining fingerprint entries account for the majority of the index. This intriguing observation inspires us to develop a hot fingerprint entry distilling approach named HID . HID segregates useless fingerprint entries from the index to improve memory utilization and bypass disk accesses. In addition, HID separates fragmented fingerprint entries to make a deduplication-based backup system directly rewrite fragmented chunks, thereby alleviating adverse fragmentation. Moreover, HID introduces a feature to treat fragmented chunks as unique chunks. This feature compensates for the shortcoming that a Bloom filter cannot directly identify certain duplicated chunks (i.e., the fragmented chunks). To take full advantage of the preceding feature, we propose an evolved HID strategy called EHID . EHID incorporates a Bloom filter, to which only hot fingerprints are mapped. In doing so, EHID exhibits two salient features: (i) EHID avoids disk accesses to identify unique chunks and the fragmented chunks; (ii) EHID slashes the false positive rate of the integrated Bloom filter. These salient features push EHID into the high-efficiency mode. Our experimental results show our approach reduces the average memory overhead of the index by 34.11% and 25.13% when using the Linux dataset and the FSL dataset, respectively. Furthermore, compared with the state-of-the-art method HAR, EHID boosts the average backup throughput by up to a factor of 2.25 with the Linux dataset, and EHID reduces the average disk I/O traffic by up to 66.21% when it comes to the FSL dataset. EHID also marginally improves the system's restore performance.

Objective To evaluate the International Classification of Health Interventions (ICHI) in the clinical and statistical use cases. Materials and Methods We identified 300 most-performed surgical procedures as represented by their display names in an electronic health record. For comparison with existing coding systems, we coded the procedures in ICHI, SNOMED CT, International Classification of Diseases (ICD)-10-PCS, and CCI (Canadian Classification of Health Interventions), using postcoordination (modification of existing codes by adding other codes), when applicable. Failure analysis was done for cases where full representation was not achieved. The ICHI encoding was further evaluated for adequacy to support statistical reporting by the Organisation for Economic Co-operation and Development (OECD) and European Union (EU) categories of surgical procedures. Results After deduplication, 229 distinct procedures remained. Without postcoordination, ICHI achieved full representation in 52.8%. A further 19.2% could be fully represented with postcoordination. SNOMED CT was the best performing overall, with 94.3% full representation without postcoordination, and 99.6% with postcoordination. Failure analysis showed that “method” and “target” constituted most of the missing information for ICHI encoding. For all OECD/EU surgical categories, ICHI coding was adequate to support statistical reporting. One OECD/EU category (“Hip replacement, secondary”) required postcoordination for correct assignment. Conclusion In the clinical use case of capturing information in the electronic health record, ICHI was outperformed by the clinically oriented procedure coding systems (SNOMED CT and CCI), but was comparable to ICD-10-PCS. Postcoordination could be an effective and efficient means of improving coverage. ICHI is generally adequate for the collection of international statistics.

Cloud-centric use of Interoperable Master Format (IMF), Open TimeLine input/output (IO) (OTLIO), and C4ID combines to transform production, post, mastering, and localization of content in today’s multiplatform and international distribution environment. By leveraging IMF, OTLIO, and other open source and standardized tools such as C4ID Cloud Identifier, a centralized cloud-based continuous master can be created. This continuous master can be constantly synchronized with work-in-progress in editing, visual effects, color grading, and sound mixing systems to provide a single centrally available point of collaboration for all parties in production, post, localization, and distribution. This method avoids the requirement for proprietary asset management or file types and allows creatives and engineers to use the tools they prefer and avoids repeatedly moving assets in and out of the cloud. Furthermore, it allows a paradigm shift in the localization and distribution master process, which can in turn allow delivery of a project almost instantly to any number of platforms, territories, and versions once it is complete. This workflow also permits sharing of conformed work-in-progress versions at any moment during the production/post/versioning life cycle without interrupting progress in any of these processes. This method also has side benefits such as total deduplication of assets, no vendor or tool lock-in for access to assets and the ability to link business systems to the content creation process, thus allowing order-to-delivery automation. Overall, this method offers benefits in terms of time, cost, operational effectiveness, and, most importantly, creativity. This benefits all types of delivery from movies to episodic, advertising, trailers, or any produced content targeting multiplatform, domestic, and international audiences. It also allows for the latest content trends such as branching storylines and audience interactive content to be stored and accessed in a nonproprietary way. The methodology described in this paper provides a step change in managing today’s content pipelines and future proofing for next-generation workflows .

Techniques of data compression involve de-duplication of data that plays an important role in eliminating duplicate copies of information and has been widely employed in cloud storage to scale back the storage capacity and save information measure. A secure AES encryption de-duplication system for finding duplication with the meaning and store up it in the cloud. To protect the privacy of sensitive information whereas supporting de-duplication, The AES encryption technique and SHA-256 hashing algorithm have been utilized to encrypt the information before outsourcing. Pre-processing is completed and documents are compared and verified with the use of wordnet. Cosine similarity is employed to see the similarity between both the documents and to perform this, a far economical VSM data structure is used. Wordnet hierarchical corpus is used to see syntax and semantics so that the identification of duplicates is done. NLTK provides a large vary of libraries and programs for symbolic and statistical natural language process (NLP) for the Python programming language that is used here for the unidentified words by cosine similarity. Within the previous strategies, cloud storage was used abundantly since similar files were allowed to store. By implementing our system, space for storing is reduced up to 85%. Since AES and SHA-256 are employed, it provides high security and efficiency.

As the rapid development of big data and the artificial intelligence technology, users prefer uploading more and more local files to the cloud server to reduce the pressure of local storage, but when users upload more and more duplicate files , not only wasting the network bandwidth, but also bringing much more inconvenience to the server management, especially images and videos. To solve the problems above, we design a multi-parameter video quality assessment model based on 3D convolutional neural network in the video deduplication system, we use a method similar to analytic hierarchy process to comprehensively evaluate the impact of packet loss rate, codec, frame rate, bit rate, resolution on video quality, and build a two-stream 3D convolutional neural network from the spatial flow and timing flow to capture the details of video distortion, set the coding layer to remove redundant distortion information. Finally, the LIVE and CSIQ data sets are used for experimental verification, we compare the performance of the proposed scheme with the V-BLIINDS scheme and VIDEO scheme under different packet loss rates. We also use the part of data set to simulate the interaction process between the client and the server, then test the time cost of the scheme. On the whole, the scheme proposed in this paper has a high quality assessment efficiency.

Secure data deduplication, as it can eliminate redundancies over encrypted data, has been widely developed in cloud storage to reduce storage space and communication overheads. Among them, the convergent encryption has been extensively adopted. However, it is vulnerable to brute-force attacks that can determine which plaintext in a message space corresponds to a given ciphertext. Many existing schemes have to sacrifice efficiency to resist brute-force attacks, especially for cross-domain deduplication, which is inevitably contrary to practical applications. Moreover, few existing schemes consider protecting the message equality information (i.e., whether two different ciphertexts correspond to an identical plaintext). To address the above challenges, in this paper, we propose an efficient and privacy-preserving big data deduplication scheme for a two-level multi-domain architecture. Specifically, by generating a random tag and a constant number of random ciphertexts for each data, our scheme not only ensures data confidentiality under multi-domain deduplication but also resists brute-force attacks. By allowing only the agent and cloud service provider to perform intra-deduplication and inter-deduplication, respectively, our scheme can protect the message equality information from disclosure as much as possible. Detailed security analysis shows that our scheme achieves privacy-preservation for both data content and the message equality information and data integrity while resisting brute-force attacks. Furthermore, extensive simulations demonstrate that our scheme significantly outperforms the existing competing schemes, especially the computational cost and the time complexity of the duplicate search.

Many countries and organizations have been sharing a vast amount of remote sensing (RS) images across the Internet via their portals to promote the flourishing development of RS applications. These portals provide the data discovery ability to the public with necessary ways. Spatial range query is one of them to enable data discovery on RS images by their spatial extents. This article first analyzes the technical challenges in building a spatial index based on the key-value model. Then, this article presents how our proposed adaptive geographic meshing and coding method solves these challenges. The proposed method transforms the spatial extent of RS images to key-value-based indexing records to ensure spatial proximity as well as a low ratio of data duplication in indexing records. This article also shows how the proposed query selectivity strategy improves the performance of spatial range queries on indexing records. The strategy determines the order of two essential operations in a spatial range query, namely, the data deduplication and the spatial relationship computation. Finally, we implement a prototype system based on HBase and develop two MapReduce-based algorithms to speedup the process of the index construction and the spatial range query. Extensive experiments on a real dataset demonstrate that our proposed method owns a better query performance as well as a lower space overhead than competitors to handle spatial range queries on RS images.

Deploying deduplication for primary storage is a challenging task in view of random access patterns of I/O requests and the requirement of quick response time. Existing deduplication approaches designed for primary workloads are locality based with the assumption that I/O requests follow the locality principle. However, primary workloads in cloud systems need not necessarily follow the locality principle, and hence, the existing methods for deduplication are likely to exhibit poor response time. To provide a solution to this challenging problem, we propose and implement a hybrid deduplication system (HDS), a block-based partial deduplication system with similarity-based indexing. The proposed system applies deduplication in the background to decrease the latency and also aims at decreasing the data fragmentation. It applies similarity-based indexing to reduce high number of metadata lookups arising out of random access patterns of the requests. HDS for primary workloads is simulated in the Linux environment using three different types of FIU traces, and the effectiveness of the system is compared with full deduplication based on the parameters—metadata access overhead, average segment length, and response time. The experimental results show that the system has performed consistently better in reducing the metadata overhead and increasing the average segment length for all three sets of I/O trace data.

Data deduplication is a crucial technique for packing data and reducing duplication when transferring data. It is widely used in the cloud to restrict the usage of capacity memory and aids in transmission capacity sparing. Before redistributing data, an encryption mechanism is used to ensure the integrity of sensitive data during the deduplication process. The SHA algorithm is being used to save data in text format. To generate the security bits, padding is appended to the text. In de-duplication, it calculates the hash, i.e. hexadecimal number, string, and integer data. Hash-based de-duplication is the implementation of whole file hashing to the entire file. Text data’s hash values are considered to as feature properties. In contrast to traditional deduplication solutions, clients that transfer data to the cloud certify duplication inside the cloud data. In virtualization, both limiting primary memory size and memory blockage are considered important bottlenecks. Memory deduplication identifies pages with the same content and merges them into a single data file, reducing memory usage, memory parceling, and improving execution. In cloud storage, the MPT is used to deduplication so that it is used in single copies of the same data for different data owners. If any data users try to replicate the same data, it will be mapped and related to the archive data, implying that the data can’t be stored away. To ensure cloud data security, encryption techniques are used to encrypt data throughout deduplication procedures and prior to outsourcing cloud data.

Cloud envisioned cyber--physical systems (CCPS) is a practical technology that relies on the interaction among cyber elements like mobile users to transfer data in cloud computing. In CCPS, cloud storage applies data deduplication techniques aiming to save data storage and bandwidth for real-time services. In this infrastructure, data deduplication eliminates duplicate data to increase the performance of the CCPS application. However, it incurs security threats and privacy risks. For example, the encryption from independent users with different keys is not compatible with data deduplication. In this area, several types of research have been done. Nevertheless, they are suffering from a lack of security, high performance, and applicability. Motivated by this, in this article, we propose a message lock encryption with neVer-decrypt homomorphic encRyption (LEVER) protocol between the uploading CCPS user and cloud storage to reconcile the encryption and data deduplication. Interestingly, LEVER is the first brute-force resilient encrypted deduplication with only cryptographic two-party interactions. We perform several numerical analysis of LEVER and confirm that it provides high performance and practicality compared to the literature.

Cloud based healthcare computing have changed the face of healthcare in many ways. The main advantages of cloud computing in healthcare are scalability of the required service and the provision to upscale or downsize the data storge, collaborating Artificial Intelligence (AI) and machine learning. The current paper examined various research studies to explore the utilization of intelligent techniques in health systems and mainly focused into the security and privacy issues in the current technologies. E-Healthcare is an emerging field of medical informatics, referring to the delivery of health services and information using the Internet and related technologies. Rendering efficient storage and security for all data is very important for cloud computing. Securing and privacy preserving of data is of high priority when it comes to cloud storage. E-Healthcare is the most important source in the healthcare society. E-healthcare system is now being popularized globally. Implementing the E-healthcare system will have more advantages such as online services for teleconsultation (second medical opinion), e-prescription, e-referral, telemonitoring, telecare etc. E-healthcare system provides high level of security and cost-effective use of patients records, information and communication in support of healthcare and health related issues.

In cloud storage, public auditing is a more popular data integrity verification technique since it allows users to delegate auditing tasks to a fully trusted third-party auditor (TPA). However, it ...

This study focuses on challenges faced during optimization of the performance of multiple users under encrypted data storage. Traditional single storage instance schemes are much concerned about the application scenarios rather than data sensitivity. This paper provides an optimization function for analyzing the performance parameters of developed queueing system. On the other hand, simulation results are further examined in order to optimize the average waiting time, average queue length, and total number of data packets. In addition, deduplication check provides more storage by eliminating duplicate files, and the remaining files further get encrypted by two cryptographic algorithms, namely advanced encryption standard (AES) and Rivest–Shamir–Adleman (RSA), respectively. Moreover, this paper emphasises performance of data in terms of storage and security; on the same basis, its comparison and implementation has been done. At last, conclusion is given.

Since network storage services achieve widespread adoption, security and performance issues are becoming primary concerns, affecting the scalability of storage systems. Countermeasures like data auditing mechanisms and deduplication techniques are widely studied. However, the existing data auditing mechanism with deduplication cannot solve the problems such as high cost and reliance on trusted third parties in traditional approaches, and it also faces the problem of repeated auditing of data shared by multiple-tenant. This article proposes a blockchain-based deduplicatable data auditing mechanism. We first design a client-side data deduplication scheme based on bilinear-pair techniques to reduce the burden on users and service providers. On this basis, we achieve a trustworthy and efficient data auditing mechanism that helps to check data integrity by using both the blockchain technique and bilinear pairing cryptosystem. The blockchain system is used to record the behaviors of entities in both data outsourcing and auditing processes so that the corresponding immutable records can be used to not only ensure the credibility of audit results but also help to monitor unreliable third-party auditors. Finally, theoretical analysis and experiments reveal the effectiveness and performance of our scheme.

The security is uppermost issue of any organisation. There are multiple alternatives for assuring the security. We have focussed on a security system and requirements of data centre of cloud using the technique hybrid cryptography. The system includes the encryption and decryption of data using various algorithms like AES, DES, etc. and Client Server Architecture. In addition, there is a technique called Deduplication is a technique for eliminating duplicate copies of repeating data. Deduplication of encrypted data is very useful in order to make a cloud service successful.

Data transmission and retrieval in a cloud computing environment are usually handled by storage device providers or physical storage units leased by third parties. Improving network performance considering power connectivity and resource stability while ensuring workload balance is a hot topic in cloud computing. In this research, we have addressed the data duplication problem by providing two dynamic models with two variant architectures to investigate the strengths and shortcomings of architectures in Big Data Cloud Computing Networks. The problems of the data duplication process will be discussed accurately in each model. Attempts have been made to improve the performance of the cloud network by taking into account and correcting the flaws of the previously proposed algorithms. The accuracy of the proposed models have been investigated by simulation. Achieved results indicate an increase in the workload balance of the network and a decrease in response time to user requests in the model with a grouped architecture for all the architectures. Also, the proposed duplicate data model with peer-to-peer network architecture has been able to increase the cloud network optimality compared to the models presented with the same architecture.

In the digital era, people often call the era of the industrial revolution 4.0 demanding speed and accuracy in all fields. Moreover, the speed and accuracy in delivering and receiving information for decision making so as not to compete with competitors. The presentation of information is not only fast, accurate, it must also be easily understood by the person receiving the information. One business entity that must also take advantage of this technological development is a cooperative. In the cooperative PT. Spindo Karawang Factory transaction activities still have to be processed in Microsoft Excel, so that data duplication or inconsistency often occurs, data updates are not updated, there is no data backup and data that has not been integrated with other systems. The purpose of this research is to increase the effectiveness and efficiency of the transaction process in the cooperative. The research method used in this research is the System Development Life Cycle (SDLC) with the waterfall approach. The result of this research is the cooperative invoice information system of PT. Spindo Karawang Factory is web-based which is expected to be able to improve the process in the cooperative to be more effective and efficient.

For cloud storage service vendors, balancing the client-perceived IO performance and the self-perceived space cost is always one of the standing challenges. When applying deduplication techniques for the cloud storage systems, the demand for optimizing such tradeoff becomes more pressing. Enabling deduplication decreases the storage space cost, whereas the IO performance will be somewhat affected due to extra processing overhead and data fragmentation. In this article, we address this challenge by proposing MUSE , a MU ti-tiered and S LA-driv E n deduplication framework for cloud storage systems. First, we propose a novel notation of Dedup-SLA (deduplication-oriented service level agreement). With different levels of quantified performance/space-cost combinations, the Dedup-SLA serves as a refined service quality protocol between service vendor and customer. Second, MUSE adopts multi-tiered deduplication that orchestrates several combinational forms of deduplication into multiple tiers with varied “deduplication strength”. Third, we implement a mechanism called dynamic deduplication regulation (DDR) to adjust the deduplication behavior during runtime. MUSE’s deduplication behavior is periodically switched between tiers according to the predefined Dedup-SLA and instant system status. We conduct comprehensive experiments to compare MUSE with several other types of deduplication schemes. The results demonstrate that MUSE significantly optimizes the IO-performance/space-cost balance compared to other schemes, hence delivering higher deduplication service quality for deduplication-enabled cloud storage systems.

Benefiting from cloud computing and mobile devices, a huge number of media contents, such as videos are shared in mobile networks. Although scalable video coding can be utilized to provide flexible adaptation, the cloud poses a serious threat to media privacy. In this paper, we propose a privacy-preserving multi-dimensional media sharing scheme named SMACD in mobile cloud computing. First, each media layer is encrypted with an access policy based on attribute-based encryption, which guarantees media confidentiality as well as fine-grained access control. Then, we present a multi-level access policy construction with secret sharing scheme. It ensures that the mobile consumers who obtain a media layer at a higher access level must satisfy the access trees of its child layers at the lower access level, which is compatible with the characteristics of multi-dimensional media and also reduces the complexity of access policies. Moreover, we introduce decentralized key servers to achieve both intra-server and inter-server deduplication by associating different access policies into the same encrypted media. Finally, we conduct experimental evaluation on mobile device and cloud platform with real-world datasets. The results indicate that SMACD protects media privacy against cloud media center and unauthorized parties, while incurring less computational and storage cost.