问题描述
当尝试编写一些并行代码时,我不断遇到以下错误。我发现了一些解决方案,但是它们都涉及锁定,而我不希望这样做。我有什么办法可以解决这个问题?
pub trait PermBrute {
fn quadgram( &self,max_len: usize,ciphertext: &String ) -> Vec<usize> {
let mut vec : Vec<(f64,Vec<usize>)> = Vec::new();
let results = Arc::new(Mutex::new(vec));
let mut threads = vec![];
for i in 0..*cpuS {
threads.push( thread::spawn({
let clone = Arc::clone(&results);
let text = ciphertext.clone();
move || {
// some code here
let hold = self.decrypt( )
// some more code here
let mut v = clone.lock().unwrap();
v.push(best_key);
}
}));
}
for t in threads {
t.join().unwrap();
}
let lock = Arc::try_unwrap(results).expect("Lock still has multiple owners");
let mut hold = lock.into_inner().expect("Mutex cannot be locked");
// do some stuff with hold and return out
return out;
}
fn decrypt( &self,ciphertext : &String,key : &Vec<usize>) -> String;
}
error[E0277]: `Self` cannot be shared between threads safely
--> src/ciphers/cipher.rs:131:27
|
108 | fn quadgram( &self,ciphertext: &String ) -> Vec<usize> {
| - help: consider further restricting `Self`: `where Self: std::marker::Sync`
...
131 | threads.push( thread::spawn({
| ^^^^^^^^^^^^^ `Self` cannot be shared between threads safely
|
= help: the trait `std::marker::Sync` is not implemented for `Self`
= note: required because of the requirements on the impl of `std::marker::Send` for `&Self`
= note: required because it appears within the type `[closure@src/ciphers/cipher.rs:140:17: 164:18 text:std::string::String,start_len:usize,end_len:usize,count:usize,start_points_clone:std::vec::Vec<usize>,local_self:&Self,end:usize,clone:std::sync::Arc<std::sync::Mutex<std::vec::Vec<(f64,std::vec::Vec<usize>)>>>]`
解决方法
使用rayon条板箱,可以使用并行迭代器技术来完成。
pub trait PermBrute {
fn quadgram(&self,max_len: usize,ciphertext: &String) -> Vec<usize> {
let mut vec: Vec<(f64,Vec<usize>)> = Vec::new();
let mut threads = vec![];
let best_keys: Vec<_> = (0..*CPUS)
.into_par_iter()
.map(|i| {
// some code here
// you can access `ciphertext` here directly without copying
todo!();
// some more code here
best_key
})
.collect();
// do some stuff with best_keys and return out
return out;
}
fn decrypt(&self,ciphertext: &String,key: &Vec<usize>) -> String;
}
,
我花了一些时间来修改您的代码,以便我可以在锈蚀操场上对其进行测试。这是修改后的源代码:
use std::sync::{Arc,Mutex};
use std::thread;
pub trait PermBrute {
fn quadgram( &self,ciphertext: &String ) -> Vec<usize> {
let mut vec : Vec<(f64,Vec<usize>)> = Vec::new();
let results = Arc::new(Mutex::new(vec));
let mut threads = vec![];
for i in 0..10 {
threads.push( thread::spawn({
let clone = Arc::clone(&results);
let text = ciphertext.clone();
move || {
// some code here
let hold = self.decrypt( &String::new(),&vec![] );
// some more code here
let mut v = clone.lock().unwrap();
// v.push(best_key);
}
}));
}
for t in threads {
t.join().unwrap();
}
let lock = Arc::try_unwrap(results).expect("Lock still has multiple owners");
let mut hold = lock.into_inner().expect("Mutex cannot be locked");
// do some stuff with hold and return out
// return out;
unimplemented!()
}
fn decrypt( &self,ciphertext : &String,key : &Vec<usize>) -> String;
}
首先,您可以通过以下方式限制Self
:
pub trait PermBrute : Sync {
然后,rustc开始困扰生命周期:
(错误太长,然后我使用playground)
this post应该回答您的问题。简而言之,thread
是生成的背景,而rustc仍然是愚蠢的,不考虑您的join
。有类似Arc<Self>
或AtomicPtr<Self>
之类的解决方法。
更新
让我们以一个最小的例子开始:
use std::thread;
fn try_to_spawn() {
let x: String = "5".to_string();
let j = thread::spawn(|| {
println!("{}",x.len());
});
j.join().unwrap();
}
在这里,rustc
说:
error[E0373]: closure may outlive the current function,but it borrows `x`,which is owned by the current function
--> src/lib.rs:5:27
|
5 | let j = thread::spawn(|| {
| ^^ may outlive borrowed value `x`
6 | println!("{}",x.len());
| - `x` is borrowed here
|
...
help: to force the closure to take ownership of `x` (and any other referenced variables),use the `move` keyword
|
5 | let j = thread::spawn(move || {
| ^^^^
这里rustc
抱怨借来的x
的生命周期。 rustc
认为:由于产生了一个线程并且该线程将在后台运行,因此它可以在函数try_to_spawn
退出之前或之后终止,因此x
可能在x.len()
获得时悬而未决。被执行。
但是显然,我们join
在函数末尾插入了线程,而我们的x
的生命周期肯定足够长(当然,从人类的角度来看,'static
的生命周期不是必需的)看法)。但是,rustc
仍然太愚蠢以至于无法理解人类,并且它对我们的join
并不了解!!
可以将x
移至结束而不是借用它。但是,以后将无法使用x
。要以“安全”的方式解决问题,请使用Arc<String>
:
use std::thread;
use std::sync::Arc;
fn try_to_spawn() {
let x: Arc<String> = Arc::new("5".to_string());
let x_clone: Arc<String> = x.clone();
let j = thread::spawn(move || {
println!("{}",x_clone.len());
});
j.join().unwrap();
println!("{}",x.len());
}
但是Arc
有开销。为了避免生命周期检查,可能要使用指针*const String
或*mut String
-原始指针不是Send
/ Sync
且不能被传送到{{1} },要通过线程之间的指针共享资源,必须使用thread
(here是关于制作指向AtomicPtr
的原始指针的讨论。)
回到问题所在,Send + Sync
(类型为self
)又如何呢?当然,这也是参考!而且&Self
也无法弄清其“真实寿命”:
rustc
产生错误信息:
use std::thread;
use std::sync::Arc;
struct S {}
impl S {
fn try_to_spawn(&self) {
let j = thread::spawn(|| {
self.do_something();
});
j.join().unwrap();
}
fn do_something(&self) {
}
}
该错误看上去与先前的生命周期错误不同,但与您的代码中发生的错误更相似。要解决此问题,请再次使用error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/lib.rs:8:31
|
8 | let j = thread::spawn(|| {
| _______________________________^
9 | | self.do_something();
10 | | });
| |_________^
|
...
:
Arc<Self>
或使用 fn try_to_spawn(self: Arc<Self>) {
let j = thread::spawn(move || {
self.do_something();
});
j.join().unwrap();
}
:
AtomicPtr<Self>
这有效但是很丑。而且我还建议使用诸如use std::thread;
use std::sync::atomic::AtomicPtr;
use std::sync::atomic::Ordering::Relaxed;
struct S {}
impl S {
fn try_to_spawn(&self) {
let self_ptr: AtomicPtr<Self>
= AtomicPtr::new(self as *const Self as *mut Self);
let j = thread::spawn(move || {
unsafe {
self_ptr.load(Relaxed) // *mut Self
.as_ref() // Option<&Self>
.unwrap() // &Self
.do_something();
}
});
j.join().unwrap();
}
fn do_something(&self) {
}
}
之类的板条箱来执行并行计算。不过,我仍然希望这个答案对您有帮助,因为您要手动创建rayon
。