问题描述
我创建了一个重叠的 WriteFile 方案,其中一个线程填充环形缓冲区并推进头指针并调用 WriteFile,另一个线程监视来自 OVERLAPPED 写入的事件以推进尾指针。逻辑行为符合预期,但文件大小没有增加,它保持不变,只是从位置 0 开始覆盖。我通过将增量值写入写入文件的内存中来测试这一点,并且索引数据增加,但是它继续写入有效的 fseek(0)。
这是代码。
首先,我以 GENERIC_WRITE 和 FILE_FLAG_OVERLAPPED 的形式打开文件,然后创建 8 个事件,每个我要写的页面一个。每页为 256KBytes。我最初只使用一个事件,但我想验证事件数量不是问题。
void
FileWriter::open(string fn)
{
cout << "opening file " << fn << endl;
m_file_handle = CreateFileA(
fn.c_str(),GENERIC_WRITE,NULL,CREATE_ALWAYS,FILE_FLAG_OVERLAPPED,NULL);
if (m_file_handle == INVALID_HANDLE_VALUE)
{
throw runtime_error("Unable to create FileWriter file handle");
}
for (size_t i = 0; i < MAX_OVERLAPPED_WRITES; ++i)
{
m_events[i] = CreateEvent(NULL,TRUE,FALSE,NULL);
}
}
现在在另一个线程中,每当缓冲区填满(8 x 256K 页)时,我都会调用此 queue_page
函数。 m_ov
是一个包含 8 个重叠结构的数组。在此调用之前,m_pages[][] 充当环形缓冲区,并且 m_pages[m_head][] 前进,将 m_pages[old page][] 向下发送到磁盘。
void
FileWriter::queue_page(unsigned page,unsigned len)
{
ZeroMemory(&m_ov[page],sizeof(OVERLAPPED));
m_ov[page].hEvent = m_events[page];
if (!WriteFile(
m_file_handle,(LPCVOID)m_pages[page],len * sizeof(float),(LPOVERLAPPED)&m_ov[page]))
{
DWORD err = GetLastError();
if (err != ERROR_IO_PENDING)
{
cout << "GetLastError() = " << err << endl;
throw runtime_error("Failed to write overlapped");
}
}
}
跟随异步写入和移动循环缓冲区尾指针的线程很简单:
void
FileWriter::wait(DWORD msec)
{
DWORD ret = WaitForMultipleObjects(MAX_OVERLAPPED_WRITES,m_events,msec);
if (ret < MAXIMUM_WAIT_OBJECTS)
{
unsigned page = ret - WAIT_OBJECT_0;
if (page < MAX_OVERLAPPED_WRITES) {
ResetEvent(m_events[page]);
}
cout << "Page write completed " << ret << " page=" << page << endl;
m_tail = (m_tail + 1) & 0x7;
}
}
经检查,环形缓冲区工作正常且没有溢出,但输出文件始终为 256KBytes。
不知道如何调试正在发生的事情,或者我错过了什么。
解决方法
@RbMm 的回答,字节访问文件需要调用者在重叠结构中设置 Offset/OffsetHigh,根据这个:https://docs.microsoft.com/en-us/windows/win32/api/minwinbase/ns-minwinbase-overlapped
修复看起来像这样:
void
FileWriter::queue_page(unsigned page,unsigned len)
{
ZeroMemory(&m_ov[page],sizeof(OVERLAPPED));
m_ov[page].hEvent = m_events[page];
m_ov[page].OffsetHigh = (m_file_offset >> 32) & 0xFFFF'FFFF;
m_ov[page].Offset = m_file_offset & 0xFFFF'FFFF;
if (!WriteFile(
m_file_handle,(LPCVOID)m_pages[page],len * sizeof(float),NULL,(LPOVERLAPPED)&m_ov[page]))
{
DWORD err = GetLastError();
if (err != ERROR_IO_PENDING)
{
cout << "GetLastError() = " << err << endl;
throw runtime_error("Failed to write overlapped");
}
}
m_file_offset += len * sizeof(float);
}